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ABSTRACT 


The purpose of the research is to examine and evaluate the emergence of the 
International Organization for Standardization, commonly referred to as ISO, quality 
standard for the year 2000 as it applies (past and present) to the Department of Defense 
(DoD) quality management system (QMS) in procurement. In particular, the researcher 
will examine the new standard and its utility for DoD procurement, focusing on changes 
from the previous ISO 9000 series. Leading defense industry quality managers and 
Defense Contract Management Agency Quality Assurance Managers will provide insight 
to the thesis. In order to understand the present and future of DoD Quality Assurance and 
Management, the thesis will briefly look at past quality assurance policy from DoD and 
the ISO. Likewise, this research will explore issues and concerns that contracting officers 
and program managers in Government will now need to become familiar with as they 
execute ISO 9000:2000. The effort will emphasize how ISO 9000:2000 fits within the 
realm of DoD procurement and what organizations need to do in order to achieve 


excellent products in a total quality management environment. 
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I. INTRODUCTION 


A. BACKGROUND 

This research will examine and evaluate the emergence of the International 
Organization for Standardization, commonly referred to as ISO, quality standard for the 
year 2000 as it applies (past and present) to the Department of Defense (DoD) quality 
management system (QMS) in procurement. In particular, the researcher will examine the 
new standard and its utility for DoD procurement, focusing on changes from the previous 
ISO 9000 series. Leading defense industry quality managers and Defense Contract 
Management Agency Quality Assurance Managers will provide insight to the thesis. In 
order to understand the present and future of DoD Quality Assurance and Management, 
the thesis will briefly look at past quality assurance policy from DoD and the ISO. 
Likewise, this research will explore issues and concerns that contracting officers and 
program managers in Government will now need to become familiar with as they execute 
ISO 9000:2000. The effort will emphasize how ISO 9000:2000 fits within the realm of 
DoD procurement and what organizations need to do in order achieve excellent products 


in a total quality management environment. 


The past decade has brought marked change in the way DoD conducts 
acquisition. A culture versed in military specifications, regulations, and standards now 
turns its focus toward better understanding the global marketplace and adapting the 
“new” economy culture to DoD procurement. One such change is in DoD management 
of quality. MIL-Q-STD 9858, once the standard by which all DoD contractors’ quality 


assurance programs were measured, is now the least preferred by DoD. 


The February 14, 1994, memorandum “Use of Commercial Standards in DoD” 
from then Under Secretary of Defense (USD) John M. Deutch opened and encouraged 
DoD acquisition and contracting professionals to “use commercial standards” to their 
maximum extent in all applicable acquisition matters. One memorandum began the 
change of converting the DoD acquisition culture of specifications in design and quality 
standards to specifications of performance and best commercial practices. Federal 


Acquisition culture continues to evolve just as the ISO 9000 QMS. Departure from MIL- 


Q-9858 represents a significant change for the DoD Acquisition workforce. Subsequent 
memorandums by Secretaries of Defense Perry and Cohen, as well as the Under 
Secretary of Defense (USD) Acquisition Technology and Logistics (AT&L) Jacques 
Gansler, championed the adoption of commercial practices. A part of this change was the 
move from MIL-Q-9858, Quality Assurance, and other DoD quality assurance guidance 
to commercial standards like ANSI 9000 and the ISO 9000 Series of Quality Assurance 
guidelines. These changes define and mark the Single Process Initiative introduced by 


the Office of the Secretary of Defense in 1995 (23). 


The move to commercial quality management standards (QMS) has not brought a 
watershed of high quality procurements. Many of the same weaknesses that existed 
under MIL-Q-9858 remain. Registered ISO 9000 organizations can and do produce 
nonconforming products. Inadequate quality assurance programs exist under ISO. 
DCMA’s role remains, as it was prior to the Single Process Initiative and subsequent 
acceptance of commercially acceptable quality management systems, to provide quality 
audit, oversight, and surveillance. Discouraging MIL-Q-9858 has not yielded the 
efficiencies and effectiveness once thought possible under non military standards. ISO 
9000 is not the panacea of QMS for the DoD or the civilian sector. One reason is DoD 
procurement is not an exact replica of civilian sector procurement. DoD has numerous 
stakeholders, often unable to develop a requirements consensus. DoD use of public funds 
associated with congressional representation and constituents do not reflect the bottom 
line philosophy of corporate America. High risk is associated with weapon systems to 
greater degree than with private system or product development. These are a few of the 


factors bearing on difficulties with QMS application in DoD. 


The dynamic environment of business and information management continues to 
change industry and DoD practices. On December 15, 2000, ISO revised their quality 
standard eliminating the five previous standards and replacing them with two. The ISO 
standard is followed by much of the industrialized world and in many aspects codifies 
international quality management practices among industrialized countries. While ISO 
9000 is a better QMS in many ways and has proven its worthiness over MIL-Q-9858, the 


acquisition environment is increasingly complicated and complex. 


Firestone, a supplier of Ford for tires, represents a recent example of ISO 
9000:1994 weaknesses where correlations can be drawn between the DoD and the 
commercial sector. Firestone, a QS 9000 registered corporation, provided non 
conforming tires for the Ford Explorer. The non-conforming tires are linked to numerous 
accidents of the Explorer. The link resulted in Explorer occupant injury and a lack of 
public trust in Ford and Firestone products. The example brings attention to one of the 
reasons for the new ISO 9000 standard. The ISO recognized, as did many of its members, 
ISO 9000 must change and adapt to some its weaknesses. One major company observed 
that documenting one’s processes and ensuring that the documented processes are 
followed, as required by the ISO standard, could result in an [SO-certified company 
producing excellent but useless concrete lifejackets (35). That particular company went 
on to develop its own quality system that placed emphasis on continuous process 
improvement and customer satisfaction, areas in which many felt the ISO standard was 


lacking (35). 


Has ISO 9000:2000 addressed the weaknesses of the previous ISO 9000:1994 
version in the dynamic environment of weapon systems acquisition? How effective is 
DoD, specifically DCMA, in properly evaluating companies using ISO 9000, and how 
can they become more effective under the new ISO 9000:2000 version? In what ways 
has and will DoD benefit from the ISO QMS, or not benefit? In short, what are ISO 
9000:2000 effects, implications, and results? 


Modern quality management systems are proactive. The idea is to design quality 
and build quality into the product instead of reacting or inspecting it in, the previous DoD 
quality assurance focus (30). No single aspect in the execution of services or 
manufacturing is more important than that of the quality of the service or item 
manufactured. The marketplace and DoD rely on the quality of products and services 
received. DoD quality managers, those responsible for quality management, and those 
who are beneficiaries of quality hold great responsibility to the public trust, especially in 
the procurement of weapon systems, due to weapon system cost, and more importantly, 
to the security of our nation and the safety of our military members. DoD knowledge and 
experience in the area of quality management systems (QMS) protects that public trust. If 


the reliance is broken, the public’s trust in the product or service is broken. The services 
e) 


provider or manufacturer’s quality systems are subject to question when the public trust 
is broken. The object of this research is to explore the issues I have outlined. 
B. OBJECTIVE AND RESEARCH QUESTIONS 

The scope will include: (1) a review of commercial and Federal quality 
management practices, (2) an in-depth review of ISO 9000 and DoD quality management 
systems, (3) a survey and evaluation of commercial and DoD quality management 
systems focusing on ISO 9000, and (4) a study of current DoD quality assurance 
practices. The thesis will conclude with recommendations for implementing the new ISO 
9000:2000 into Defense Contract Management Agency (DCMA) audits and program 


office evaluation of contractors. 


The primary research question is what implications and effects will ISO 
9000:2000 Quality Management System have on the DoD acquisition community? 
Subsidiary research questions include: 


° What is ISO 9000? 


e What is the Government’s requirement for a quality management system 
and how has it affected the DoD and defense contractors? 

e How does ISO 9000:1994 fit the Government’s quality requirements and 
will ISO 9000:2000 be a better fit? 

e What are the key conclusions and recommendations that might be drawn 


regarding the ability relationship ISO 9000:1994 and ISO 9000:2000 to 
meet the Government’s quality requirements? 


C. METHODOLOGY 


The methodology used in this research will consist of the following steps. 


e Conduct a literature search of books, magazine articles, CD-ROM 
systems, Internet, library, and other resources 

e Conduct a review of commercial and Federal quality standards through a 
review of past and present quality practices 

e Conduct a thorough review of ISO 9000 and DoD quality management 
systems 

e Examine the current capabilities and limitations of DoD and ISO 9000 


quality standards 


e Compare and contrast TQM, ISO 9000, and DoD quality management 
systems addressing the advantages and disadvantages of each 


D. 


Evaluate the benefits and cost of ISO 9000 certification, both the 1994 and 
2000 versions 


Question both commercial and DoD practitioners of quality management 
systems 


Identify essential elements of DoD and commercial practices in quality 
management and how best they can be applied under DCMA, and program 
offices 


Identify areas of weakness and potential changes to DoD quality 
management guidance in comparison 


Prepare and conduct analysis of ISO 9000, commercial, and Federal 
procurement quality management regulation and guidance 


Recommend changes to current DoD practices in quality management 
audit, evaluation and surveillance. 


ORGANIZATION OF THE STUDY 


The appendices include a comprehensive glossary of abbreviations and acronyms, 


a comparison to ISO 9000 versions 2000 and 1994 and MIL-Q-STD 9858, and the 
current DCMA Quality Assurance Checklist. 


The research effort is organized into five chapters and eight appendices. Chapter 


I presents the thesis, outlining the objective, methodology, and organization of the 


research effort. Chapter II highlights ISO 9000:1994 and 2000 background, history, and 


standards. 


requirements. 


Chapter [II presents and discusses the DoD quality management system 


Chapter IV provides perspective of quality assurance managers from 


Government and DoD contactors and analysis of the implications and effects of ISO 


9000:1994 and 2000. I close with recommendations and conclusions in Chapter V. 
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I. IS09000 


A. INTRODUCTION 

We will begin the research paper by providing some background on quality 
management systems. The background will briefly cover some key guidelines and 
systems based on performance and commercial standards from the old MIL-Q-9858 and 
Government defined standards, and provide an overview of the ISO: 9000 series quality 
standards that DoD has used as a framework for evaluating its own suppliers for the past 
seven years. The paper will then follow with an analysis and discussion of the salient and 
pertinent matters from the issues above. The main effort of the research paper is to focus 


on DoD management of quality systems and their relationship with ISO 9000. 


A quote by Mark Schaeffer, Deputy Director, Systems Engineering, 
(DTSE&E/DDSE) sets the tone and framework for the thesis. 


Quality is not just another interesting subject, nor is this just another 
conference. Quality is one of those overarching [sic] considerations that 
we all believe in—we all strive for—but that too few seem to really 
understand (48). 


Quality is not a new concept discovered in the Ast century. Consumers have 
understood the essence of quality since goods and services were bought and sold from 
earliest times. Quality is most simply defined and recognized by the user, but as there are 
many definitions of quality, there are many stakeholders in the effort of quality assurance 
and management. Joseph Juran defines the meaning of quality in two areas of critical 
importance (28). 

Quality means those features of products, which meet customer needs and 

thereby provide customer satisfaction. In this sense, the meaning of 

quality is oriented to income. The purpose of such quality is to provide 
greater customer satisfaction and, one hopes, to increase income. 

However, providing more and/or better quality features usually requires an 


investment and hence usually involves increases in costs. Higher quality, 
in this sense, usually “costs more.” 


Quality means freedom from deficiencies — freedom from errors that 


require doing work over again (rework) or that result in field failures, 
customer dissatisfaction, and customer claims, and so on. In this sense, the 


7 


meaning of quality is oriented to costs, and higher quality usually “costs 

less.” (28) 

Quality, like beauty, has varied definitions and is subject to the perspective of the 
beholder. Simply put quality refers “to all the features of a product (or service) which are 
required by the customer.” Quality management is defined as what the organization does 
to ensure that its products conform to the customer’s requirements (7). Quality 
management and assurance are explored in this thesis, focusing on Juran’s second 


definition of quality. DoD has traditionally oriented procurement towards cost. 


Quality today is preventionoriented and process-driven. This shifts the total 
responsibility for quality from the quality professionals to everyone in the organization. 
No longer a "stovepipe" discipline, quality must be an integral element of engineering, 
manufacturing, and coding of a product. Quality must be integral to the way we (DoD) do 
business. (50) 

B. HISTORY 

The US Military has had a strong influence on international quality standards. 
The Department of Defense began developing quality standards in the mid-1950s using 
techniques and procedures used during the industrial build up during World War II (36). 


The military played an early role in quality standards. As early as 1450 B.C. the 
Egyptians used a form of measurements and inspection for swords. In the Middle Ages 
craftsmen both manufactured and inspected their work. Poor craftsmanship led to no 
business. Industrialization separated the craftsmen from manufacture and inspection. 
Honoré Le Blanc developed a pattern for musket manufacture for the French. 
Manufacture and quality inspection were separate (15). The eighteenth century created 


continued specialization. 


In 1798, Thomas Jefferson introduced one of the earliest uses of U.S. military 
contracting and quality assurance when the newly formed nation contracted with Eli 
Whitney to manufacture 10,000 muskets. The muskets’ construction called for 
interchangeable parts. Due to problems associated with the new technology of mass 
produced weapons (special machining, training and inspection) and the Government’s 


discovery of frequent poor quality, it took Eli Whitney nearly ten years to deliver the 


10,000 muskets originally contract for delivery in two years. Industrialization continued 


to separate quality responsibility from the hands of the producer during the early years of 
the U.S. (15,35) 


During the years of the then War and Navy Departments and prior to World War 
Two, Government inspectors practiced 100 percent inspection of manufactured goods 
and material. The early view concerning quality was that it was the War Department’s 
responsibility and duty to assure and validate the quality of the goods and services 
provided to the services. Assurance was gained through the imposition of rigid standards, 
exacting specifications, and the Government performing inspections throughout the 


production process. (33) 


The twentieth century ushered new thought on quality assurance. Early pioneers 
of quality control like Walter Shewhart, Howard Dodge, George Edwards, and W. 
Edwards Deming expanded thought and technology in quality discipline. Shewhart led 
his team of Bell Labs quality pioneers in the development and use of Statistical Quality 
Control (SQC). SQC went beyond mere inspection and the focus created a method to 
identify problems in the process of manufacturing and production (15). World War II 
marked a shift to sampling methods as a means of ensuring quality. Due to the large 
amounts of material being procured for the war effort it became impractical and costly for 
the War and Navy Departments to continue reliance on 100% inspection to assure 


quality. 


The Government found that it could achieve a high degree of confidence in 
product quality by using sampling methods. Statistical sampling plans and methods were 
developed and used as additional methods to assure quality while reducing Government 
reliance on 100% inspection methods. (33,35) This era began the U.S. military use of 
stringent standards on its suppliers. The War Production Board offered training in the 
new quality control process, much like the Defense Acquisition University offers courses 
today (15). Dr. Deming himself was one of the lecturers. The aim was to train people 
how to maintain the quality of delivered product while substantially reducing the 
inspection time. This also meant a substantial reduction in the number of Government 


inspectors. As a result, the Army introduced sampling inspection procedures in 1942 to 
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ensure and maintain quality without 100% inspection. By 1945, several similar plans 
were available across the Services. During the 1950s, the DoD published three military 
standards for inspection sampling, and in 1963 one of these documents became an ABCA 


(America, Britain, Canada, and Australia) Standard. (50) 


During the early 1950s, as weapon systems continued to increase in complexity, 
the Air Force pioneered requirements for a contractor quality program. These 
requirements were aimed at major contractors engaged in the development, production, 
and selected service tasks for aircraft and missile systems. Following the Air Force 
example, the other Services initiated similar requirements for their products of similar 
complexity. The OSD staff brought all parties together in the late 1950s to establish a 
coordinated approach on quality. These organizations developed and issued Military 


Specification MIL-Q-9858, Quality Program Requirements. (50) 


These standards are what we currently call Military Quality Standards (MIL-Q- 
STD) (15). In the late 1950’s the DoD developed the MIL-Q-9858 standard (titled 
“Quality Program Requirements”), which subsequently was used as the model for British 
Standard BS-5750, released in 1979 (15,33). In the early 1960’s, the DoD developed a 
second standard, the MIL-I-45208 (titled “Inspection System Requirements’’). 


One of the models for the ISO 9000 series of internationally accepted quality 
standards model was BS-5750. A great degree of commonality exists between DoD 
MIL-Q 9858 and the ISO 9000 series (15). The reader may review Appendix B (ISO to 
MIL-Q Cross Reference) for a comparison of MIL-Q 9858, ISO 9000:1994, and ISO 
9000:2000. 


MIL-STDs, however, were not influencing the vast majority of commercial 
manufacturing in the post-war era. Expanded commercial production called for even 
greater standardization and consumer need for higher levels of quality. The Japanese 
using, ironically, Henry Ford’s book, My Life and Work, and heavily influenced by 
Deming developed the methods of total quality management (TQM) shifting quality 
responsibility back to the worker. The U.S. began employing TQM philosophy in the 
early 80s (15,50). During this time in the late 1980’s - early 1990’s DoD shifted to a 


“management systems” approach to ensure quality. 
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The International Organization for Standardization (ISO), founded in 1947, 
embarked to prepare international standards relating to quality assurance techniques and 
practices during the international shift in management philosophy. A technical committee 
was approved and worked under the title of Quality Assurance ISO/TC 176. The ISO 
created the world’s first internationally accepted “generic quality management standards” 
and outlined in the ISO 9000 family of standards with the first editions published in 1987 
(47). 


When ISO/TC 176 embarked on the development of generic quality management 
standards for worldwide application, it was able to take advantage of a substantial base of 
national experience, notably in the United Kingdom and in Canada. In the United 
Kingdom, the BS 5750 standards were well on their way to broad acceptance and, in 
Canada, a series of national standards known as CAN-Z299 were also widely used. Other 
countries with well-developed quality management practices, such as Japan, also took a 
keen interest in the work of the new committee. In addition, experience of military 
quality assurance specifications, such as the NATO AQAP series (see Appendix C 
(NATO AQAP)), enriched the sources from which TC 176 was able to draw. (7) 


The ISO 9000 series of International Quality Standards are an outgrowth of 
efforts by the European Committee for Standardization and the ISO. The formation of 
the European Union in 1992 provided the major thrust in harmonizing the nineteen 
different European country standards into one. The ISO 9000 series standards were 
released by the International Organization for Standardization in 1987 and were updated 
in 1994. ISO released its newest version in late 2000. The International group adopted 
the name ISO for the standards, from the Greek word “iso” which means “equal” (1,35). 
(The reader will see the use of ISO to be interchangeable with International Organization 
for Standardization (ISO) as it is used in this paper.) During the latter part of the 1980s, 
DoD and many defense contractors discussed the possibility of using the new ISO 9000- 
ANSI/ASQC 90 series of quality standards instead of military specifications. However, 
DoD did not see any significant benefit in changing to the 1987 version of the ISO 9000 


series. 


11 


During this period in the early 90’s, a shift occurred in the methods DoD used for 
procurement. Numerous examples of Executive Branch and DoD policy and guidance 
document the base of our current Government and DoD procurement policy. A partial list 
and a brief synopsis of these examples are provided in Appendix E (Executive and DoD 
Policy Guidance). 

C. UNDERSTANDING THE ISO QUALITY SYSTEM 

1. ISO Basics 

ISO 9000 is actually a family of standards, referred to under a generic title for 
convenience. The family consists of standards and guidelines relating to management 
systems, and related supporting standards on terminology and specific tools, such as 
auditing (the process of checking that the management system conforms to the standard). 
ISO 9000 is primarily concerned with "quality management," meaning what the 
organization does to ensure that its products conform to the customer's requirements. ISO 
9000 is also concerned with the way an organization goes about its work, and not directly 


the result of this work. (52,53). 


Nevertheless, the way in which the organization manages its processes is 
obviously going to affect its final product. In the case of ISO 9000, it is going to affect 
whether or not everything has been done to ensure that the product meets the customer's 
requirements. However, ISO 9000 is not a product standard. The management system 
standards in these families state requirements for what the organization must do to 
manage processes influencing quality (ISO 9000). The philosophy is that these 
requirements are generic. No matter what the organization is or does, if it wants to 
establish a quality management system, then such a system has a number of essential 


features that are spelled out in ISO (3,52). 


The ISO 9000 family of standards represents an international consensus on good 
management practices with the aim of ensuring that the organization can time and time 
again deliver the product or services that meet the client's quality requirements. These 
practices have been distilled into a set of standardized requirements for a quality 
management system, regardless of what the organization does, its size, or whether it's in 


the private, or public sector. Full-time standardization experts do not develop ISO 
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standards. The business sectors most interested in implementing the eventual standards 


are the ones who provide experts to develop the standards. (3,7) 


The primary quality organizations representing U.S. interests are the following. 
The American National Standards Institute (ANSI) represents over 250 US organizations 
that write standards, and represent the U.S. in ISO. The American Society for Quality 
(ASQ), formerly ASQC, coordinates with ISO and directs ISO activities in the U.S. on 
behalf of ANSI. The Registrar Accreditation Board (RAB), an ASQ _ subsidiary 
corporation, accredits quality systems registrars in the USA. The RAB also performs 
initial evaluations of prospective registrars and issue certificates. (32,34) 

Zz Generic Management System Standards 

ISO 9000 provides a standardized framework for taking a systematic approach to 
managing business processes (an organization's activities) so that they consistently turn 
out product conforming to the customer's expectations. ISO 9000 lays down what 
requirements a quality system must meet, but does not dictate how they should be met in 
an organization — which leaves great scope and flexibility for implementation in different 
business sectors and business cultures...as well as different national cultures (52). This 
flexibility allows for organizations to develop their own unique ways to implement ISO 


standards. 


The vast majority of ISO standards are highly specific to a particular product, 
material, or process. However, both ISO 9000 and ISO 14000 are known as “generic 
management system standards.” Generic means that the same standards can be applied to 
any organization, large or small, whatever its product — including whether its "product" is 
actually a service — in any sector of activity, and whether it is a business enterprise, a 
public administration, or a Government department (7,52). Management system refers to 
what the organization does to manage its processes, or activities. To be really efficient 
and effective, the organization can manage its way of doing things by systemizing it. This 
ensures that nothing important is left out and that everyone is clear about who is 


responsible for doing what, when, how, why and where (15). 


The organization should carry out auditing of its ISO 9000-based quality system 


itself to verify that it is managing its processes effectively — or, to put it another way, to 
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check that it is fully in control of its activities. This is a self-audit. A second party audit is 
one where an organization invites its clients to audit the quality system in order to give 
them confidence that the organization is capable of delivering products or services that 
will meet their requirements. DCMA’s audits and surveillance are second party audits. 
Organizations may engage the services of an independent quality system certification 
body to obtain an ISO 9000 certificate of conformity. This last option has proved 
extremely popular in the market place because of the perceived credibility of an 
independent assessment. It may thus avoid multiple audits by the organization's clients, 
or reduce the frequency or duration of client audits. The certificate can also serve as a 
business reference between the organization and potential clients, especially when 
supplier and client are new to each other, or far removed geographically, as in an export 
context. (6,52) 

3. Certification, Registration and Accreditation 

Certification, registration and accreditation are often misunderstood and 
misleading and are directly related to misconceptions of the ISO quality management 
system and what it can or cannot provide to an organization and its customers towards 
achieving a quality product or service. In many individual’s minds, there is a linkage 
between the organization “ISO” and the concept of quality system certification and 


registration. 


The assessment of a quality system against the requirements of one of the ISO 
9000 standards and the subsequent issuance of a certificate to confirm that it is in 
conformance with the standard's requirements is variously referred to in different 
countries as certification or registration. Likewise, the bodies that issue ISO 9000 
certificates — “certification bodies” — are referred to in some countries as "registration 
bodies" or "registrars." In the United States, “registrars” is the commonly used term. 


Again, these different appellations refer to the same type of body (7). 


A final point on terminology concerns "ISO 9000 certification." In fact, "ISO 
9000 certification" means certification against ISO 9001, ISO 9002 or ISO 9003 (1994 
version) or ISO 9001 and 9004 (2000 version). The generic term "ISO 9000 
certification” is much more convenient than alternatives such as "ISO 9001/2/3" or "ISO 


900x" certification. This is why ISO 9000 certification has entered into common usage, 
14 


and is employed in this thesis as in other literary products and common language. 
However, only an actual "ISO 9000 certificate" will specify the standard against which 


the quality system in question has been assessed and found to be in conformance. (11,52) 


ISO itself does not carry out assessments to check that users are in conformity 
with the requirements of the standards. Conformity assessment — as this process is known 
— is a matter for suppliers and their clients in the private sector, and of regulatory bodies 
when ISO standards have been incorporated into public legislation (7). The concept of 
ISO 9000 certification and registration was developed by the private sector. These 
activities are conducted by independent commercial and private organizations, which 


have no direct affiliation with ISO. (32) 


A point, which logically follows from the above remarks, is that it is false to 
describe a company as "ISO-certified," "ISO-registered," or to use phrases such as "ISO 
certification,” "ISO certificates" and "ISO registration." ISO operates no system for 
assessing the conformance of organizations’ management systems with standards in the 
ISO 9000 family. ISO itself does not carry out ISO 9000 audits and awards no certificates 
attesting to conformity with the standards. There is no such thing as "ISO certification," 
or "ISO registration," whether in relation to ISO 9000, ISO 14000, or any other ISO 
standard. ISO 9000 auditing and certification are carried out independently of ISO by 
certification bodies under their own responsibility. (7,11) 

D. ISO 9000:1994 

The ISO-9000:1994 series consists of five standards, along with some 
complementary standards, e.g., ISO-8402: 1994 Quality Management and Quality 
Assurance — Vocabulary (4). 


The five standards are: 
e TISO9000-1: Quality Management and Quality Assurance Standards - Part 


e ISO9001: Quality Systems - Model for Quality Assurance in Design, 
Development, Production Installation and Servicing 


e TSO9002: Quality Systems - Model for Quality Assurance in Production, 
Installation and Servicing 


e TSO9003: Quality Systems - Models for Quality Assurance in Final 
Inspection and Test 
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e 1SO9004: 1994 Quality Management and Quality System Elements - Part 
1: Guidelines 


The ANSI/ASQC documents covered under ANSI/ASQC Q-9000 represent 
different levels of quality requirements outlined as follows: (4,8) 

° ANSI/ASQC-Q9001 "Quality Systems - Model for Quality Assurance in 
Design/Development, Production, Installation, and Servicing" 


° ANSI/ASQC-Q9002 "Quality Systems - Model for Quality Assurance in 
Production and Installation" 


° ANSI/ASQC-Q9003 "Quality Systems - Model for Quality Assurance in 
Final Inspection and Test" 


° ANSI/ASQC Q-9001, Q-9002 and Q-9003 are the U.S. equivalents and 
equal to the international quality standards ISO 9001, ISO 9002, and ISO 
9003, respectively 
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Figure 1. | ISO Spheres of Influence, [After Ref. 35]. 


ISO 9001 is the most far-reaching standard of all three Quality System Models 
9001, 9002 and 9003, see Figure 1 (35). ISO 9001 covers a contractor’s quality program 
throughout its entire lifecycle, from design, through production, installation and 
maintenance. Of the three ISO 9000 documents, ISO 9001 is the most reflective of MIL- 
Q-9858 (41). ISO 9002 addresses production, final inspection and test, and installation. 
ISO 9003 covers only final inspection and test. The following excerpts from ISO-9001 
will familiarize the reader with its twenty elements. In many cases, information from one 
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element affects the other elements. In this way, each element influences the others. In the 
ISO 9000 series documents (and in many quality systems based on this standard), the 
word “supplier” refers to the company following the quality standard. ISO purports 


managing suppliers over managing supplies. (1) 


ISO 9000 provides a framework for a quality system and contains twenty 


elements as depicted below: (4,35) 


1 Management Responsibility 11 Control of Inspection, Measuring & Test Equipment 
2 Quality System 12 Inspection & Test Status 
3 Contract Review 13 Control of Nonconforming Product 
4 Design Control 14 Corrective & Preventive Action 
5 Document & Data Control 15 Handling, Storage, Packaging, Preservation & Delivery 
6 Purchasing 16 Control of Quality Records 
7 Customer Supplied Product 17 Internal Quality Audits 
8 Product Identification & Trace ability 18 Training 
9 Process Control 19 Servicing 
10 Inspection & Testing 20 Statistical Techniques 
Table 1. The 20 Elements of ISO 9000:1994, [After Refs. 4,35]. 


E. ISO 9000:2000 


iL Catalyst for Change 


The major reasons for the year 2000 revisions of the standards include: 


e Emphasizing the need to monitor customer satisfaction 
e Meeting the need for more user-friendly documents 
e Assuring consistency between quality management system requirements 


and guidelines 


e Promoting the use of generic quality management principles by 
organizations, and enhancement of their compatibility with ISO 14001 


e Emphasis on continual improvement 

Extensive surveys were performed by the ISO commission tasked with ISO 
revision, ISO/TC 176, on a worldwide basis to understand the needs of all users of the 
quality management system standards. The new revisions take into account previous 
experience with quality management system standards (1987 and 1994 editions) and 
emerging insights into generic management systems. ISO 9000:2000 purports a closer 


alignment of quality management systems with the needs of organizations and better 
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reflects the way organizations run their business activities. ISO's rules of procedure (the 
ISO/IEC Directives) also specify that standards be periodically revised to ensure that 


those standards are current and satisfy the needs of the global community. (14) 


ISO 9001:2000 aims at guaranteeing the effectiveness (but not necessarily the 
efficiency) of the organization. For improved organizational efficiency, however, the best 
results can be obtained by using the new ISO 9004:2000 in addition to ISO 9001:2000. 
The guiding quality management principles of ISO 9004:2000 are intended to assist an 
organization in ISO 9001:2000 principles like continual improvement, which should lead 
to efficiency throughout the organization. (48) 


Efficiency refers to the “capacity to produce results with minimum expenditure of 
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time, money, or materials.” Efficiency focuses on the input-output ratio. To be efficient 
is to do things well, to attend to the internal organization by refining, routinizing, 
formalizing, elaborating on existing knowledge, and making short-run improvements. 
Effectiveness, on the other hand, is defined as “productive results.” The focus is on 
doing the right thing and an absolute level of input or outcome determines efficiency. 
Effectiveness and efficiency comes from an understanding and interpretation of the 
external environment as it signals what ongoing adaptations in goals, outputs, and 


outcomes are required. (49) 


ISO established a 3 year “transition” period during which accredited certification 
to the 1994 standards and ISO 9001:2000 may continue to coexist. This “transition 
period” will end on 15 December 2003. By that date, all organizations wishing to retain 
accredited certification will have to bring their quality management system to compliance 
with ISO 9001:2000. (52) 

2. ISO 2000 Family 

The ISO 9000 family previously contained more than 20 standards and 
documents. This proliferation of standards was a particular concern of ISO 9000 users 
and customers. To respond to this concern, the ISO 9000:2000 family consists of four 
primary standards supported by a considerably reduced number of supporting documents 
(guidance standards, brochures, technical reports, and technical specifications) compared 


to previous editions. To the extent possible, the key points in the past documents were 
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integrated into the four primary standards, and sector needs addressed while maintaining 


the generic nature of the standards. The four primary standards are (25): 


e ISO 9000: Quality management systems — Fundamentals and vocabulary 

e ISO 9001: Quality management systems — Requirements 

e ISO 9004: Quality management systems — Guidance for performance 
improve ment 

e ISO 19011: Guidelines on quality and/or environmental management 


systems auditing (to be published) 
ISO 9001 and ISO 9004 create a “consistent pair’ of quality standards. The 
primary aim of the "consistent pair" is to relate modern quality management to the actual 
processes and activities of an organization, including the promotion of continual 


improvement and enhancement of customer satisfaction. (48) 


The quality management system described in the revised standard is based on 
quality management principles that include the "process approach" and "customer focus." 
ISO 9001:2000 and ISO 9004:2000, both, apply a process approach. The "20 element" 
structure of ISO 9001:1994 has been replaced by this process-based quality management 
system (PQM), which is shown schematically in Figure 2, below. (28, 48) For more 
information about the process management approach and how it is applied to ISO 


9000:2000, review Appendix F (ISO Process Management Approach). 
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Figure 2. Model of a Process Based Quality Management System, [From Ref. 48]. 


ISO recommends the greatest value be obtained when the entire family of 
standards is used in an integrated manner. ISO 9001:2000 is the first level of 
performance. ISO 9004:2000 practices are implemented to make the quality management 
system increasingly effective in achieving business goals. ISO 9001:2000 and ISO 
9004:2000 have been formatted as a consistent pair of standards to facilitate their use in 
combination (48). Although they are "stand alone” standards, they provide synergy when 
used together, increasing organizational efficiency and effectiveness (25). The reader 
may at this time review Appendix D (ISO Family), which provides details of the ISO 
9000:2000 Family. 

3 ISO 2000 Activities and Disciplines 

ISO 9001:2000, like its 1994 predecessor, is used by organizations seeking to 
establish a management system that provides confidence in the conformance of their 
products to established or specified requirements. It is now the only standard in the ISO 
9000 family against whose requirements a quality system can be registered to by an 


external agency (48). 
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There are five sections in the standard that specify activities, often referred to as 
the five disciplines of ISO 9000. Product realization describes the activities used to 
supply products; quality systems may exclude the parts of the product realization section 
that are not applicable to an organization’s operations. The product realization section is 
the only area where exclusion is an option. The requirements of the other four sections 
are: quality management system, management responsibility, resource management and 
measurement, analysis and improvement. These four apply to all organizations. The five 
sections of ISO 9001:2000 define a consistent approach to provide product that meets 
customer requirements. ISO 9004:2000 is used to extend the benefits obtained from ISO 
9001:2000 to all stakeholders (48). The ISO 90001 sections are organized into eight 
standard clauses. (26,27) They are: 


e Section 1 — Scope 

e Section 2 - Normative Reference 

° Section 3 - Terms and Definitions 

e Section 4 - Quality Management System 

e Section 5 - Management Responsibility 

e Section 6 - Resource Management 

e Section 7 - Product Realization 

e Section 8 - Measurement, Analysis, and ImprovementOf the eight clauses 


of the new standard, sections 4 through 8 are the most important to the QMS. These 
clauses actually comprise the “heart” of the quality management system. Each one of 
these five main clauses has a number of sub clauses which support the concept identified 
in the main clause. (32) The reader may find additional perspective on the structure of 
ISO 9000:2000 by reviewing Appendix G (ISO 9000:2000 Re-structure). 

4. Quality Management Principles 

Eight quality management principles provide the basis for the revised ISO 
9000:2000. Senior management, as a framework to guide their organizations towards 
improved performance, can use these principles. The principles contained in the new ISO 


9000 standards are more “aggressive” in nature than the previous version. 


The changes to the ISO standard series require the user to be more proactive. The 
principles help to define and validate the quality of the work, goods, or services they 
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provide. The principles concentrate on customer focus, top-level management, 


involvement and continuous improvement. (26,32) 


The list below provides the standardized descriptions of the principles as they 
appear in ISO 9000:2000 and ISO 9004:2000. ISO states that the goal and benefit of the 
eight quality management principles, that underlie the ISO 9000:2000 series, is to form a 
basis for performance improvement and organizational excellence beneficial to the 
organization’s quality management system. (26,32) 

e Principle 1 -- Customer focus 


Organizations depend on their customers and therefore should understand 
current and future customer needs, should meet customer requirements 
and strive to exceed customer expectations. 

e Principle 2 -- Leadership 

Leaders establish unity of purpose and direction of the organization. They 
should create and maintain the internal environment in which people can 
become fully involved in achieving the organization's objectives. 

e Principle 3 -- Involvement of people 

People at all levels are the essence of an organization and their full 
involvement enables their abilities to be used for the organization's 
benefit. 

e Principle 4 -- Process approach 

A desired result is achieved more efficiently when activities and related 
resources are managed as a process. 

e Principle 5 -- System approach to management 


Identifying, understanding and managing interrelated processes as a 
system contributes to the organization's effectiveness and efficiency in 
achieving its objectives. 


e Principle 6 -- Continual improvement 


Continual improvement of the organization's overall performance should 
be a permanent objective of the organization. 


® Principle 7 -- Factual approach to decision making 


Effective decisions are based on the analysis of data and information. 


e Principle 8 -- Mutually beneficial supplier relationships 
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An organization and its suppliers are interdependent and a mutually 
beneficial relationship enhances the ability of both to create value. 


ISO states that the introduction of the quality management principles is to address 
the proliferation of the 20 elements of ISO 9000:1994 and to better align with 
contemporary philosophy and objectives of most quality award programs and best 


management practices (48). 
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Il. DOD QUALITY ASSURANCE AND MANAGEMENT POLICY 


A. INTRODUCTION 

DoD QMS Policy is covered in a series of Federal and Defense Department 
Regulations, Policy, Directives, and Guidance. The Federal Acquisition Regulation 
(FAR), the Defense Federal Acquisition Regulation Supplement (DFARS), and the 
Department of Defense Directive 5000.2-R (DODD 5000.2-R) encompass U.S. DoD 
policy on acquisition and associated quality requirements. One of the primary agencies 
charged with executing the policy on quality management and assurance is the Defense 
Contract Management Agency (DCMA). DCMA provides guidance through the use of 
the DCMA Onebook. DoD QMS policy and the current policy relationship to ISO 9000 
is based on a series of Executive guidance documents published in the 1990s, which are 


summarized in Appendix E (Executive and DoD Policy Guidance). 


The change in DoD policy from military specifications to commercially 
acceptable quality standards resulted from concerns from industry that standards like 
MIL-Q-9858 requiring significant documentation in areas such as quality policies and 
procedures test results and analysis, manufacturing problems and proposed corrective 
actions were an excessive and unnecessary burden on industry. Contractors and many 
DoD employees suggested that much of the documentation added little value to the 
manufacturing and thus would consume contractor resources that could otherwise be 
spent on activities that add value (16). Many contractors were also using dual QMS: 


MIL-Q-9858 and a commercial QMS such as ISO 9000. 


DoD recognition of ISO 9000 pleased industry but concerns of DoD quality 
assurance auditors continuing to use MIL-Q-9858 were high. Industry desire for a new 
“higher level quality standard” policy addressing the shift in DoD procurement quality 
assurance began to arise. DoD addressed this issue by inactivating MIL-Q-9858 on 1 
October 1996 and redesigning DFARS 246. In doing this DoD pushed contractors and 
the Defense Contract Management Command (now DCMA) to focus on commercial 
quality assurance standards and guidelines. DCMA oversight developed measures to 


streamline existing oversight process to reduce contractor compliance assets by shifting 
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emphasis from audits/inspections to reviews of contractor processes (12). ISO 9000 
allowed DoD to take better advantage of commercial best practices, lessons learned and 
quality assurance and management techniques and procedures. Contractors no longer had 
to carry the resource burden of managing two or more quality programs or systems and 
industry could react more quickly to quality innovations and then put them into action. 
DoD realized two benefits: cost savings from DoD contractors using commercial higher 
quality standards, and the cost savings associated with no longer maintaining the MIL-Q- 
STD regulations and guidelines. This chapter focuses on the DoD relationship towards 
ISO 9000. 
B. DCMA POLICY 

1. Contractual Elements 

Quality products and services are fundamental to successful military operations, 
system development and production. The quality of products or services is determined by 
the extent that they meet (or exceed) requirements to satisfy customer(s) at an affordable 
cost. The Department of Defense (DoD) acquisition policy is to emphasize development 


of quality products through the design of the product and its associated processes (22). 


The DCMA mission prior to award is to assist in constructing effective 
solicitations, identify performance risks, assist in selecting capable contractors, and 
ensure easily administered contracts. Post award, DCMA ensures product, cost, and 
schedule compliance, including onsite surveillance and program specific processes. (19) 
DoD contract quality requirements fall into four general categories (8): 

e Contracts for commercial items (FAR 46.202- 1) 

e Government reliance on inspection by contractor (FAR 46.202-2) 

e Standard inspection requirements (FAR 46.202-3) 

e Higher- level contract quality requirements (FAR 46.202-4) 

Contracting activities select the appropriate category based upon the technical 
requirements, the complexity, the commercial/Federal application of the components 
associated with the overall, and the criticality of the associated system (22). When the 
contracting activity determines a higher-level contract quality requirement is necessary 
and appropriate, contractors may specify their preferred documented quality/inspection 


system. This system could be based on (32,33): 
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More specifically FAR Part 46.401 prescribes (8): 


Government contract quality assurance shall be performed at such times 
(including any stage of manufacture or performance of services) and 
places (including subcontractors' plants) as may be necessary to determine 
that the supplies or services conform to contract requirements. Quality 
assurance surveillance plans should be prepared in conjunction with the 
preparation of the statement of work. The plans should specify- all work 
requiring surveillance; and the method of surveillance. 


Each contract shall designate the place or places where the Government 
reserves the right to perform quality assurance. If the contract provides for 
performance of Government quality assurance at source, the place or 
places of performance may not be changed without the authorization of 
the contracting officer. If a contract provides for delivery and acceptance 
at destination and the Government inspects the supplies at a place other 
than destination, the supplies shall not ordinarily be reinspected at 
destination, but should be examined for quantity, damage in transit, and 
possible substitution or fraud. Government inspection shall be performed 
by or under the direction or supervision of Government personnel. 
Government inspection shall be documented on an inspection or receiving 
report form or commercial shipping document/packing list, under agency 
procedures (see Subpart 46.6). Agencies may prescribe the use of 
inspection approval or disapproval stamps to identify and control supplies 
and material that have been inspected for conformance with contract 
quality requirements. 


e The appropriate International Organization for Standardization (ISO) 9000 
standard 
e The appropriate American National Standards Institute (ANSI) / American 


Society for Quality (ASQ) Q9000 standard 
e The appropriate NATO AQAP standard (see Appendix C) 


° A canceled Government standard with supplements (i.e., Mil-Q-9858, 
Mil-I-45208) 

e A process control system that meets other recognized industry (but non 
ISO/ANSI/ASQ) standards 

e A process control system that is equivalent to or better than the ISO 9000 
standard 


Allowing contractor flexibility to attain the highest level of quality through 
contractor selected, and subsequently, DoD approved QMS is the goal of SPI. DoD 
encourages contractors who currently have contracts citing the MIL-Q-9858A and MIL-I- 


45208A requirements to consider transitioning into commercial quality system standards 
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via the SPI (22,23). The quality system referenced in a contract is a contractual 
requirement and the contractor is responsible for instituting and maintaining that quality 
system. In most cases, the DCMA becomes involved in contract administration support 
by request of the procuring contracting activity that issued the contract (8,22). It is a 
common misconception that DoD requires ISO registration. Contractors have the 
flexibility to adopt any quality assurance and management system so long as it addresses 


the customer’s requirements. 


FAR Part 46.202-3-90 prescribes the insertion of a clause covering manufacturing 
process controls and in-process inspection. The clause introduces the possibility of ISO 
9000 insertion in the RFP. The contractor then chooses ISO 9000 or other option. This 
allows offerors to provide information that will enable the source selection team to assess 
risk of performance, and to identify those practices the offeror deems important to the 
project in terms of the risk mitigation. The risk assessment will be based on the ability of 
the offeror to carry out program requirements and evidence of demonstrated 
effectiveness. 

2 Government Responsibility for Product Quality 

The Government bears the responsibility of determining whether the standard is 
implemented properly and is adequate for the risk involved. That determination usually 
involves a quality system audit performed by DCMA. The Government does not require 
any contractor to become certified or registered to ISO standards. FAR Subpart 46.2 
outlines guidance concerning contract quality requirements. Specifically, 46.202-4 
prescribes higher-level contract quality requirements. Requiring compliance with higher- 
level quality standards is appropriate in solicitations and contracts for complex or critical 
items or when the technical requirements of the contract require control of work 
operations, in-process controls, and inspection; or attention to organization, planning, and 
work instructions. In the RFP Section L, Instructions to Offerors and Section M, 
Evaluation Factors for Award, cover the insertion of higher-level quality system clauses 
as outlined in FAR 46.311. The objective is for the supplier to define and mitigate the 


manufacturing process risks associated with the acquisition cycle (4,9). 


Based on the thrust toward the use of commercial standards and specifications, 


ISO 9000 has had an ever-increasing importance to Government contracting. On 18 
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November 1996, the DCMA (then DCMC) issued Memorandum No. 96-73, Quality 
Systems Evaluation (Policy), which recognized ISO 9000 and the associated registration 
process. An abstract of this letter follows: 
DCMA/CAO is to perform ISO 9000 audits on selected contractors if the 
contractor is moving toward implementation of ISO 9000. Audit results 


are only revealed to the customer (PMO, Buying Command) and the 
contractor (35). 


In support of the use of ISO 9000, DCMA/CAO will perform full audits 

when requested by the buying command or if no information is available 

as to the adequacy of a contractor’s quality system the CAO will verify 

third party registration data, including past audits and the traceability of 

the registrar to the Registration Accreditation Board (RAB). DCMA will 

accompany the third party auditors if concurred with by the contractor. 

When participating with an audit, the CAO auditors shall audit 

management responsibility, corrective and preventative actions, and 

internal audits. 

Upon successful completion of the audit, DCMA will issue a statement of 
compliance. According to DCMA, the ISO 9000 Series has two primary roles, quality 
management and quality assurance. Two Types of ISO 9000 standards guidance 
standards and contractual standards are currently in use. DCMA representatives will see 

ISO 9000 series standards are used in four situations (35). They are guidance for 
Quality Management, contractual, between first and second parties, second-party 


approval or validation, and third-party certification or registration. 


In most cases all contracts have two “deliverables,” the quality system 
requirements and the goods or services requested. The quality system requirements are as 
much a contractual requirement as are the goods or services requested. Validation of the 
quality system is as contractually necessary as validating the serviceability of the goods 
or services. Quality Systems are a contractual requirement. ISO 9000 fulfills the 


requirements of a “higher level” quality system. 


DCMA specialists must evaluate contractor quality systems for compliance with 
contractual higher-level contract requirements. The ISO 9000 Standards are the most 
widely used non-government standards in DoD contracts. When a “higher level” quality 


system is called out in a contract, that contract has at least two deliverables - the quality 
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system and the material or services requested. Part of the cost associated with a contract 
is the cost related to the quality system. Just as we verify the goods and/or services 
contained in the contract we also must validate the contractual quality requirement cited 


in the contract. (32,34) 


The DCMA One Book, Chapter 4.4, outlines how DCMA personnel will conduct 
contractor quality systems evaluations. Quality system audits are tailored to examine 
only those quality elements directed by customers, and/or those elements where existing 
data does not provide confidence. DCMA personnel evaluate the contractor’s quality 
system for compliance using existing data from credible first, second, or third party 
audits. Sample verification or confidence in the auditing process may be used to establish 


the credibility of audits conducted by others. (8,22) 


Policy and guidance on the application of quality standards is provided in the 
FAR Part 46; DFARS Part 246; SECDEF Memorandum of June 29, 1994, subject: 
"Specifications and Standards-A New Way of Doing Business"; and USD (A&T) 
Memorandum of December 8, 1995, subject: “Single Process Initiative" (13,19,20). 
ANSI/ASQC Q-9000, and/or the ISO-9000 series standards are referenced as model 
quality management systems. Contractors are given the flexibility to respond with their 
own quality systems. DODD 5000.2-R Part 5 provides quality management guidance and 
as the others mentioned above will be referenced and further explored throughout the 


discussion and analysis. 


DCMA mission requirements have remained the same and increased in many 
areas even though DCMA manpower and budgetary resources have decreased 
significantly, from 26,500 in 1989 to fewer than 13,000 currently (19). Contracting out, 
or outsourcing, increasing public scrutiny of DoD procurement and budget, and 
commercial industry unaccustomed to having DoD as a customer have all affected the 
way in which DCMA conducts business. DCMA’s aging workforce has witnessed great 
internal and external change in DoD procurement since the late 1980s. In order to 
address these issues, DCMA uses risk planning, assigning a risk rating to suppliers and 
allocating resources to match the risk associated with the procurement. Risk planning, 


assessment, and handling are directly correlated to supplier quality assurance. The 
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DCMA One Book Section 4.2 addresses this area. DCMA risk planning is conducted 
through review of contract and customer requirements and the identification of quality 
systems, processes, and characteristics. The latter identification complements the 


principles of the ISO QMS. (45,57) 


DCMA has two tools at their disposal to balance contract insight and 
administration with the limited resources. The tools are Process Oriented Contract 
Administrative Services, or PROCAS as it is more commonly called, and Contractor 
Self-Oversight, CSO. PROCAS is the DCMA approach of prevention and improvement 
rather than correction to address and manage the contractor to Government relationship. 
Teaming and partnering between contractor, customer, DCMA, and DCAA are the 
actions to improve and analyze better quality management process. CSO is an alternative 
to direct oversight by DCMA. The DCMA and contractor sign a Memorandum of 
Agreement (MOA) covering interaction, schedule records, and changes. Clearly defined 
surveillance plans must be in place for CSO to work effectively. The surveillance plan 


dictates how the CAO will monitor the contract CSO. (54) 


The DCMA takes numerous audit approaches to guide their evaluation of a 
contractor and these approaches are used to discover underlying problems in a quality 
system affecting areas of concern for the ACO and PCO. There are varying indicators 


that result in a DCMA Audit ofa contractor’s quality system: (8) 


e Customer request 

e Inadequate contractor data to establish confidence 
e Unsatisfactory performance 

° Defective pricing 

e Internal control issues 


e TQM program efficiency 

Quality system evaluations must be accomplished based on the objective data. 
Sample verifications of key quality system elements are authorized to validate the 
credibility of reports of audits conducted by others. DCMA quality system audits focus 
on the specific elements of the quality system identified for review by the customer, or 


the elements(s) of the system where confidence in compliance is lacking. (8,22,24) 
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Generally, DCMA evaluations and audits of contractor quality systems must be 
performed using (ISO) or American National Standards Institute (ANSD/American 
Society for Quality (ASQ) 9000 series quality system models. DCMA specialists invite 
customer participation in audits of contractor quality systems. The DCMA Audit 
Checklist guides audit performance. A capable QMS is given a Statement of 
Qualification by DCMA. Whether the quality system evaluation results are positive or 
negative, the results must be provided to the DCMA customer(s), because they provide 
valuable insights into the capabilities of a contractor’s quality system. (8,24) 

C. DCMA POLICY 2000 

1. Policy 

DCMA Policy has not undergone any major overhaul since it published Quality 
Management Systems — Requirements ISO 9001:2000. The ISO standard meets the 
criteria established by FAR Part 46.202-4 as a higher-level quality assurance 
requirement. It is expected that this standard will be included in DoD contracts via FAR 
Clause 52.246-11, Higher-Level Contract Quality Requirement. In accordance with 
continuing policy, the DoD does not require accredited certification to higher-level 
quality standards, to include industry standards. DoD contractors are required to 
maintain effective quality systems in accordance with contractual requirements. DCMA 
issued Information Memorandum No. 01-142 (IM # 142) on February 13, 2001 providing 
guidance to the Commanders of Defense Contract Management Districts (CMD), 
DCMA, and Contract Management Offices (CMO). The following paragraphs focus on 
DCMA policy according to IM # 142 (24). 


Many defense contractors have based their quality management systems on the 
1994 edition of ISO 9001, 9002, or 9003. Although it is expected that many defense 
contractors will continue to base their quality systems on the 1994 editions, DCMA is 
working closely with the DoD and industry groups to devise an approach for the 
transition from the 1994 editions of ISO 9001 to the 9001:2000 version, when 
appropriate. Although DoD does not require certification, the 1994 editions of ISO 9001, 
ISO 9002, ISO 9003 and ISO 9001:2000 will co-exist for three years for certification 
purposes. DoD is working toward a streamlined process on defense contracts to simplify 
implementation of the ISO 9001:2000 when elected by a contractor, provided no increase 
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in price or fee is required. The transition process will incorporate the Single Process 
Initiative provisions contained in DFARS 211.273. Transition activities will include the 
following: (8,22,24) 


e Contractors shall notify the contracting officer in writing of this election to 
include their transition strategy; the use of established Management 
Councils, including sector and corporate councils, is encouraged and 
expected 


e If DCMA is the cognizant contract administration activity and the 
contractor elects to update their quality management system from the 1994 
edition, DCMA is authorized to monitor the transition, evaluate 
implementation, and approve and issue appropriate modifications to 
associated contracts 


e If DCMA is not the cognizant activity, the cognizant contracting officer 
will generate appropriate contract modifications 


The ISO 9001:2000 revision replaces 1994 versions of ISO 9001, 9002, and 9003 
(43). The 1994 family of ISO 9000 documents originally comprised over 27 standards. 

Three of the twenty-seven were contractual standards: ISO 9001, ISO 9002, and 
ISO 9003. In the 2000 series there will be only one contractual standard. That standard is 
ISO 9001, referred to as ISO 9001:2000.With the release of the new series, the ISO 
9001:2000 is now the only quality management system standard in the 2000 series. ISO 
9001:2000 allows the quality management system to be tailored (22). The ISO 9001 
clause, which refers to this process, is “1.2-Application.” This clause allows a level of 
flexibility for the adoption of the standard by various businesses (i.e., suppliers who 
would have previously utilized the ISO 9002 or ISO 9003 standard because of the scope 
of their business.) Clause 1.2 limits tailoring to the requirements found in clause 7. The 
quality manual must indicate if the system is tailored and indicate the tailored clauses and 
provide justification for their exclusion. Tailoring of requirements is not a new concept to 
the DoD. For many years the DoD buying commands have used tailored quality system 
requirements in satisfying their needs. Under the 1994 models, suppliers could choose the 
quality system model, which best represented the scope of the work they performed (i.e., 
design or development, production, etc.).Now there will be just one quality system 
model, ISO 9001.Obviously not all suppliers quality systems need to embrace all the 


clauses (elements) contained in that document. “Tailoring,” as it relates to the revised 
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ISO 9001 standard, simply allows suppliers to identify those quality system clauses 


(elements), which apply to their quality system (33). 


The ISO 9001: 2000 allows suppliers to apply only those specific clauses, which 
represent the scope of the work that they perform. This use of “application” however only 
applies to those requirements contained in clause 7 of the ISO 9001:2000 standard, 
Product Realization. The application can’t be used to reduce the scope of suppliers’ QMS 


(8,26,57). 


The DCMA One Book Supplier Quality Assurance, Chapter 4, discusses Quality 
System Evaluations and Audits in section 4.6.3.1.2. DCMA specialists are not to perform 
new evaluations or audits of previously acceptable ISO 9001 or 9002 contractor quality 
systems, due solely to a transition from the 1994 versions of the standards to the ISO 
9001:2000 version. Specialists should, as necessary, assess contractors’ systems 
associated with the differences between the revisions. Appendix B (ISO to MIL-Q Cross 
Reference) provides a comparison of ISO 9001:1994 and ISO 9001:2000. As is the 
normal DCMA policy for Quality System Evaluations, existing credible data must be 
used in conducting the evaluations; audits must be limited to those clauses where existing 
credible data does not provide confidence that the system is compliant. Evaluation 
results may be recorded in any convenient format, indicating how confidence was 
established for each applicable quality system clause. DCMA must notify the contractor 
in writing of the results of this assessment. These written notices must identify the 
applicable quality system standard requirements assessed (8). When assessments indicate 
significant noncompliance, the written notice must identify the exact areas of 
noncompliance and request corrective action. 

2. Implementation 

The utilization of the new ISO 9000 series standards impacts the DoD customer, 
the DCMA workforce, and the DoD supplier. Due to the coexistence of the two standards 
series for the next three years, greater attention will need to be placed in the identification 
of the ISO 9001 revision referenced in contractual documents to ascertain if the ISO 
9001:2000 requirement and system are tailored. DCMA breaks down suppliers into two 
categories when using ISO 9000 based systems, registered/certified or non 


registered/non-certified (33). Suppliers can continue to use 1994-based standards and 
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may be referenced in contracts as long as it satisfies the customer’s needs, although 
registered/certified DoD suppliers with ISO 9000 based systems are driven by 
registration requirements to adopt the new standard. Documents must reference a 
contractual standard and a specific version (1994 or 2000). If the ISO 9001:2000 system 
is tailored, DCMA is the activity assuring the tailored system will satisfy the DoD 
customer’s needs. The worldwide acceptance of ISO continues to influence 
implementation of QMS. Research external to the interview found that DCMA currently 
administers approximately 24,000 contracts with higher-level quality requirements 
produced by approximately 1,100 suppliers. Contract clause 52.246-11 requires the use 
of a higher-level quality requirement, including ISO 9001 (20). Since the adoption by 
DoD of ISO 9000 as allowable to use in the early 1990s, registered companies have 
grown from 225 to approximately 40,000 companies. As of July of 2000, nearly 340,000 
registrations of certification existed internationally (15, 25). As of 23 May 2000, ISO 
Certificates totaled 343,643 in 150 countries. The U.S. has over 30,000, Britain has over 
60,000, and Germany has over 30,000. This is an increase 21% in one year (54). A 
survey of 240 companies revealed some interesting statistics about ISO 9000:1994. (31) 


e 89 percent of certified companies and 77 percent of noncertified 
companies agree that ISO 9000 is a practical quality system for 
manufacturers. (64 percent of certified and 60 of non certified companies 
say it is practical for service providers. ) 


e 73 percent of certified companies and 67 percent of noncertified 
companies agree that ISO 9000 will save them money in the long run 


e 78 percent of certified companies and 71 percent of noncertified 
companies say it will “definitely improve quality” in the long run 


e 87 percent of certified companies and 76 percent of norcertified 
companies say ISO 9000 is “necessary to remain competitive” 


e ISO 9000 a “value-added quality system” 
Since ISO does not issue certificates of conformity to ISO 9000, there is no 
official ISO database of the number of certificates or registrations issued by independent 


auditors. The data is from an ISO-sponsored survey of its worldwide membership. (25) 
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IV. ISO 9000 FIT INTO GOVERNMENT QUALITY 
MANAGEMENT REQUIREMENTS 


A. INTRODUCTION 

How does ISO 9000:1994 fit the Government’s Quality Requirements and will 
ISO 9000:2000 be a better fit? The analysis of this question is based on the data presented 
in the previous chapters and the appendices. The scope of this chapter considers 
consistent themes found in the research. The themes are defined and the thesis writer 
presents an overall impression of those themes to establish conclusions and 


recommendations for Chapter V. 


Appendix H, Interviews of Quality Managers, provides the raw interview data, 
which serves as the basis for this chapter. The interviews present a sampling of 15 
managers, directors, or auditors of quality assurance management departments from 
companies that are currently ISO 9000:1994 registered, and from the DCMA. Contractor 
organizations were selected based on their prominent role in the defense industrial base. 
DCMA representation was selected based on their lead role in organizing and distributing 
guidance and training on ISO 9000 and expert knowledge of military quality assurance 
and management. The aggregate list of respondents’ companies is presented in Appendix 


A (List of Companies Participating in Interviews). 


The companies selected were manufacturers of hardware and/or software for the 
DoD. The scope of the thesis did not include representatives from the service industry, 
but much of the information drawn from this research may apply to the service industry 
as well. All of the respondents were involved in DoD contracts and are aware of the ISO 
9000 Standards and MIL-Q-9858/MIL-I-45208. As a result, the reader must recognize 
that each respondent brought his own biases toward ISO 9000 standards, registrars of 


ISO, and the DoD. 


Of the 13 private companies that provided interviewees, only one had obtained 
ISO 9000:2000 registration. All organizations were using an incremental approach to 
move towards ISO 9000:2000 registration directed from the corporate or enterprise level. 


Quality systems’ planning was done primarily at the corporate or enterprise level while 
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allowing respective business units and departments to plan execution and 
implementation. The incremental approach allows one business unit or department to 
take the lead role in ISO 9000:2000 implementation. The corporation’s contracted third 
party registrar then conducts an ISO 9001:2000 audit. The audit results or lessons- 
learned are distributed throughout the corporation or enterprise to all other business units 
seeking ISO registration. Those units, in turn, use the results and lessons-learned to 
develop their own ISO transition plan. The incremental approach works with only one or 
two unit audits occurring simultaneously. This allows subsequent units to seek and 


acquire a higher level of fidelity into each of their respective ISO implementation plans. 


Most of the companies maintain departments within their organization or have 
suppliers that retain MIL-Q quality standards as their QMS. All of the respondents were 
managers or directors of quality assurance or management within their organization and 
all had recent experience with ISO Registrars and DCMA. In most cases the person 
contacted was responsible for the organization’s transfer to ISO 9000:2000 and was very 
familiar with registrars of ISO and the ISO standards. Only one respondent’s 


organizational business unit decided not to meet the ISO 9000:2000 standard. 


This chapter consists of three sections, each dealing with a theme emerging from 
the interviews. These themes were: advantages and barriers, policy vs. perception, and 
the relationships between defense contractors and DoD. 

B. ADVANTAGES AND BARRIERS 

The first theme to emerge from the interviews was that of advantages and barriers. 
This section will draw from interview questions that addressed ISO 9000 registration and 
use. Of the companies interviewed, none required ISO as a condition of entry onto their 
qualified supplier list. However, ISO, to varying degrees, is a selection factor. Most 
defense contractors see ISO 9000 as advantageous for suppliers to have but fall short of 
requiring ISO 9000 or third party registration. One respondent stated, 

The imposition of the ISO registration process on suppliers could cause 

financial/technical barriers for them. To alleviate the problem we perform 


audits to ensure that the supplier has sufficient control in-place to ensure 
that product provided to us meets or exceeds our customers’ requirements. 
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The degree of confidence in the registration was based primarily on two factors, 
the confidence the prime contractor had in the registrar of the potential supplier and the 
past performance data and reputation of the supplier. In all companies interviewed, ISO 
offers a reduction in the degree of oversight required towards suppliers, dependent on 
each industry evaluation of specific contract risk (59). A common theme quoted by 
respondents is: 


Registered suppliers avoid some of the obstacles required to become part 
of approved supplier base, however, registration is not a pre-requisite. 


DoD uses ISO registration in much the same way as commercial firms. ISO 
registration is a factor to consider during risk management of contractor selection. 
Knowing that a company follows ISO, presumes that generally accepted baselines of 
quality management and assurance practices are in use. Third party registration increases 
the level of assurance by providing a level of confidence that the quality system is 
actually in use and practiced by the workforce. Since registrars are independent of ISO, 
no one registrar is identical to the others, and therefore past performance and the ability 


of the supplier continue to play the critical role in selection. In fact, many defense 
contractors and suppliers still use MIL-Q-9858 and MIL-I-45208. 


The cost and benefits of ISO 9000 registration via a second or third party registrar 
must obviously be weighed by each organization. Worldwide acceptance of ISO has 
made it, either intentionally or unintentionally, a requirement for entry into many 
markets, especially overseas markets. While ISO registration is not mandatory to 
compete for DoD contracts, ISO has influenced the quality assurance audit and 
surveillance process, not only overseas, but in the U.S. as well. The influence is reflected 
in DoD guidance. ISO 9000 is an example of a standard that meets the DoD “higher level 
quality standard” requirement. Most organizations feel ISO 9000 is necessary to compete 
internationally. Some organizations retain the position that ISO 9000 is used as a barrier 
to market entry specifically in Europe, as stated by one respondent. 

ISO is not value-added to our organization. It is more of a marketing tool 


than an improvement to our quality system. Our registrar’s advice and 
surveillance have been of little use. 
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It is difficult to deny that ISO 9000 is a barrier to entry in many markets, given its 
worldwide acceptance. The ISO barrier, whether real or perceived, is based widely on 


top management’s view of ISO’s value to the organization. 


Many maintain that ISO 9000:1994 was not a significant change from MIL-Q- 
9858 and that there is little additional benefit or cost to transition to ISO 9000:1994. The 
primary cost these respondents noted was the high price of third party registration. DoD 
does not require the third party registration but many markets do, thus adding support to 
the view that ISO 9000 is a marketing tool. One company position: 

Having previously been a MIL-Q-9858 company, the cost tradeoff 

between ISO registration activities (including internal audits and other 


internal efforts) was equally counterbalanced with the absence of frequent 
ACO/customer quality system evaluations. 


Benefits ranged from increased productivity, increased flexibility, and quality 
assurance discipline, to the absence or reduction of customer inspections provided as a 
derivative of the ISO Third Party Registration. Others were more skeptical, stating it has 
not brought a significant advantage to the organization over MIL-Q-9858 but resulted in 
costing more for the third party registration. Companies seeing ISO as beneficial view 
ISO as a catalyst to change the corporation’s quality culture: 

The most beneficial outcome of ISO is to have an outside registrar to 


validate the quality cultural changes that need to be made in our 
organization. 


What we once thought as unrelated to quality is actually related. It effects 

the whole organization. Previously we looked at the downstream process, 

ISO brings in the upstream processes that impact the product. Quality 

goes to management commitment, [and] capable processes on _ the 

manufacturing floor. ISO can bring in the discipline to change. 

To a great degree, the actual impact of implementing ISO 9000 in an organization 
depends on the importance the organization places on emphasizing ISO principles and 
discipline. The standard is not the panacea for quality problems, and without the addition 
of leadership, management, and a workforce embracing a total quality perspective; ISO 
has little chance to create a recipe for success in the organization implementing it. When 


ISO is used as a catalyst for organizational change, incorporating a TQM and process 


management approach, it can provide meaningful results in quality production, but alone, 
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without a champion and an organization to build additional processes and systems around 


ISO 9000, it will merely meet the minimum requirements. 


More specifically many saw the ISO 9000:2000 elements of product realization, 
customer satisfaction, and continual improvement as offering the most significance and 
concern by defense contractors and the DoD. Product realization is seen as a mixed 
blessing, providing both gains and drawbacks to industry and the Government. The 
concept allows for the true commercialization of the QMS; however, the contractor must 
justify to the Government exclusions sought under the product realization clause. Under 
the Standard Inspection Clause, the Government holds final approval of any exclusion. 
Excluding certain provisions not needed for contract quality assurance may provide 
reduced cost for the Government. Including un-necessary items and clauses under 
product realization may add unattended cost to a contract where no direct benefit is 


gained from inclusion of the clause. 


Respondents believed the 2000 version of the ISO 9000 series addresses 
shortcomings with customer satisfaction and continual improvement, but also commented 
that customer satisfaction and continual improvement introduce new problems and 
concerns. Concerns surround the abilities of DCMA and registrars, or any auditor’s 
ability, to make accurate measures of customer satisfaction and continual improvement as 
it applies to DoD contractors. The customer is not easily defined or recognizable in DoD 
procurement. Many stakeholders are considered customers in DoD. DCMA, DCAA, 
DLA, Systems Mangers, Program Managers, Congress, and the users alike all have 
legitimate claim to some aspect of the DoD customer. The program office is the user 
representative to defense contractors, ISO 9000:2000 has not changed that relationship, 
but all stakeholders assert some influence as customers of the defense contractor. 
Registrars and DCMA should consider the unique situation in which contractors face in 
having DoD as a customer when assessing this area. This is an area that needs to be 


worked out as ISO 9000:2000 is implemented. 


Continual improvement is another area difficult to measure in many instances, 
especially in mature products and service-based industry. For example, it is easy to see 


continual improvement in computers over the past few years, but is the same true for 
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laundry detergent or in the janitorial service? Both laundry detergent and janitorial 
service are examples of products where it is difficult to see noticeable differences in 
improvement unless one looks outside the product itself for improvements in 
management and cost effectiveness. Again it calls upon the auditor and PCO/ACO to 
properly define how these facets of ISO will be addressed and measured in the contract. 


In fact it may be in the Government’s interest to not evaluate this area at all. 


Barriers associated with ISO 9000 discussed by the respondents ranged from the 
systemic to the resources required in implementing the change. The barriers applied to 
ISO 9000:1994 as well as implementation of ISO 9000:2000. The resource areas of time, 
training, and financial cost topped the list. Systematic changes or challenges, as one 
interviewee said it, were documentation, awareness training, and the establishment of 
systems to capture required data. Across all industry interviewees concerns were voiced 
about respective registrars interpretation of ISO 9000:2000 and how future audits will 
vary and change from the previous version (1994) audits. Nearly all respondents again 
mentioned culture, which had an overriding importance relative to other factors. If the 
culture of the organization cannot adapt to progressive ways of doing business, the 
organization will cease to exist or prosper. The competitive forces of the market place 
will replace organizations that cannot adapt, evolve and change to meet ever increasing 
and technologically challenging customer requirements. The general perspective and 


consolidation of interview comments by industry and DoD is presented below: 


The initial barriers or challenges to ISO 9000:2000 certification are: 


e An initial and significant effort in mapping or restructuring one’s 
documentation system to align with the standard. 


e An initial increase in the need for employee awareness training due to 
additional requirements 


e Establishing the budget for training internal auditors and hiring a registrar. 


e Designing a system to capture evidence of improvement programs and 
upper management’s involvement. 


e Re-calibration between [company] and [their] registrar regarding some of 
the new requirements and exactly what they mean. (Based on the 1987 to 
1994 transition, the variation of interpretation reduces with time; but the 
initial differences in opinion starts out fairly wide). 


42 


e The culture of “us versus them,’ between the quality inspectors and 
production, is difficult to change. The culture change needed is the move 
to make everyone responsible for quality assurance in the organization, 
resulting in a better product. Making each one focus on their [sic] 
processes in quality management so [they] do not have the performance 
and defect problems to begin with. The goal is to make sure all the various 
partners in [the] company take responsibility and accept their function for 
the quality of what they do whether it is a process or a product. This helps 
every one understand that quality is everyone’s business. 


c. POLICY VERSUS PERCEPTION 

A large degree of misconception and misunderstanding is apparent between the 
policy of DoD QMS and ISO 9000 and the perceptions of both the DoD and defense 
contractors. This section is primarily based on interviews (see Appendix H), Quality 
Process Versus Quality Product and examines areas where disconnects exist between 
policy reality and policy perception. This section provides insight and understanding into 


that policy. 


Industry respondents were at varying stages in the ISO 9000:2000 transition 
process. Opinion within organizations varied in the degree to which the new ISO might 
effect the organization. Those who thought the new version would change the 
organization’s QMS to a geat degree seemed to be placing their hopes on continuing to 
shift organizational quality culture. Respondents saw the external force of ISO 
registration as a catalyst to make more changes to internal quality systems and move the 
organization toward more process management to enhance the quality of manufacturing. 
Dissenting opinion was that ISO changed little and that planning was focused on 
updating documentation requirements as published by respective registrars to meet the 
registration to ISO. All organizations that participated in interviews were using an 


incremental approach to registration, viewed from the corporate or enterprise level. 


Some see ISO 9000 registration as a marketing tool rather than a quality 
assurance tool or standard valuable to the organization, as mentioned in section B of this 
chapter. To some extent, it may be true. ISO Registrars are independent of ISO and as 
with any independent organization are subject to the bottom line of profit in order to 
survive. A case could be made that ISO is used as a barrier to entry in the market place; 


especially in Europe where ISO is more predominate. ISO 9000 certification is both a 
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marketing tool and a good quality management system. ISO 9000 provides no 
competitive advantage due to its worldwide proliferation. It is best described as a quality 
system baseline within industry and provides a common international version of 
generally accepted quality management standards and assurance techniques to provide 


goods or services in the global market place. 


As the reader will see in a later section on quality process versus quality product, 
meeting a process can still lead to manufacturing scrap. The process must match a set of 
industry- and organization-specific set of metrics that measure the effectiveness and 
efficiency of processes that produce an item meeting the conformity of the customer’s 
requirements. Product specific measures, organizational leadership, workforce 
experience, culture, and training all play a role in total quality. No standard can dictate 
entrepreneurial uniqueness to guide a manufacturer to it market niche. Two opposing 


views of ISO 9000:2000 are presented: 
First, the “barrier to market entry” view: 


What was unique about ISO was the registration process. It was used as a 
barrier for entry into the common market. The registration framework 
makes it successful and perpetuates the success, financially, of the 
companies that are registrars. It is a product or service the registrars 
provide for sale. ISO is a commercial product, not a significant customer 
support. 


Second, the view that ISO 9000 makes a difference: 


I see the ISO changes as significant. A heavy emphasis on continuous 
improvement, management oversight, record meeting. It places more a 
burden on contractors to document their continuous process improvement. 
I disagree that the five disciplines talked to in the new ISO were already 
built into the previous 20 elements. 


The Firestone example, introduced previously, brings attention to one of the 
reasons for the new ISO 9000 standard. ISO recognized, as did many of its members, that 
ISO 9000:1994 must change and address some of its weaknesses. One major company 
observed that documenting one’s processes and ensuring that the documented processes 
are followed, as required by the ISO standard, could result in an [SO-certified company 


producing excellent but useless concrete lifejackets (35). That company went on to 
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develop a quality system that emphasized continuous process improvement and customer 


satisfaction, areas in which many felt the ISO 9000:1994 standard were lacking (35). 


The eight QMS Principles of ISO are a solution intended to address problems 
with ISO 9000:1994: such as demonstrated processes, evidence of product conformance, 
and documentation as well as customer satisfaction and continual improvement (14,35). 
The new standard places more emphasis on management’s involvement in quality 
systems and process and requires auditors to spend more time and effort verifying 
effectiveness through interviews of people on the production floor or conducting the 
service. The shift of ISO 9000 from the 20 elements to the eight management principles 
provides an excellent framework for teaming and implementing the principles in an 
integrated approach rather than a stovepipe one. Many organizations had divided the 20 
elements into respective departments to administer, creating a stovepipe approach to ISO 
9000:1994 implementation. The eight principles, which embody the 20 elements, cause 
organizations to take an integrated approach to their performance and execution. Some 
defense contractors perceive the 20-element elimination as a needed improvement. 

The only improvement that I see in the 2000 version is that it is 

restructured to eliminate the perception that ISO 9000 is 20 separate, stand 

alone elements that “the quality guy” is responsible to ensure are met. 


Unfortunately, a large degree of vagueness in the requirements of the 
standard accompanies the new revision. 


Others see the principles and disciplines of ISO 9000:2000 as intrinsic properties 
of the 20-elements. 

What is obvious to some in the previous 20 elements is not so to others. 

What organizations see as shortcomings or weaknesses and even strengths 


of ISO is largely dependent on the culture of the organization 
implementing ISO. 


No matter the opinion, organizations that adopt ISO 9001:2000 will have to 
demonstrate that their product realization processes are effective and producing quality 
product. They will also have to demonstrate that product conformity or critical 
characteristics are acceptable (35), and must establish plans for improvement. 
Improvements must then be actually demonstrated. Contractual acquisitions for 


commercial and noncomplex items would not normally require higher-level quality 
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assurance requirements such as ISO 9001:2000. If the requirements of the quality 
management system are being fully met, then, processes are in control, product 
conformity levels are at acceptable levels, and substantial plans for continual 


improvements are in place. (4,14,35) 


Interviews and thesis research reflect that ISO 9000 is more effective when used 
with advanced quality procedures. Quality product development involves a broad range 
of quality techniques and measurement. Numerous inspection, modeling, and sampling 
techniques are in use in addition to ISO 9000 by the defense contractors interviewed. 
The quality system in use, as one respondent put it, “ must be flexible and dynamic... 
with a list of menu options or courses of action to follow to implement... in order to 


distinguish between a trend and an isolated incident (59).” 


One quality manager with an extensive software program more specifically made 


the following remark, 


We have many pre-ship quality processes. Most are based on the CMM 
Model [sic] and follow CMMI and internal standard processes [The 
respondent was referring to the Capability Maturity Model (CMM), and 
the associated Capability Maturity Model Integration (CMMI)].! We have 
a process that looks at and measures the critical paths of a product; design, 
build, manufacturing, test and function, focusing on the products’ “critical 
characteristics.” We also use break and fix, accelerated life testing to 
mature our products to measure quality level. 


On the production and service side, we have numerous management 
reviews, various avenues for customer feedback, tracking systems like 
systems defect trends. All of these procedures are linked to our 
organization’s failure review board involving all product players 
(stakeholders). Suppliers and customers are a part of the entire process 
and manufacturing relationship. An output is our Quality System Report 
Card providing our process to improve. 


A clear distinction is made between an organization with the ability to produce a 


quality product and one that actually does produce a quality product. An organization 


can meet registration requirements, proving it has a system of quality, but can and may 


The respondent was referring to the Capability Maturity Model (CMM), and the associated Capability 
Maturity Model Integration (CMMI). The Software Engineering Institute at Carnegie-Mellon University 
developed CMM and CMMI. DoD 5000.2-R requires software-intensive Major Defense Acquisition 
Programs (MDAPs) to meet certain requirements specified in the CMM. 
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still produce material that is nonconforming or even defective. A popular analogy can be 
made between a surgical operation and a quality system and how measures of the system 
or process may not be appropriate. 
Hospitals are expected to produce quality but performance based on 
quality alone is never measured, in part because the meaning of the term 
quality is bothelusive and disputed [as the researcher has noted in Chapter 


Il]. For instance, physician claims that quality stems from using the 
“correct procedure,” no matter what the outcome (41). 


A surgical procedure or process may be high quality but if it does mt match the 
end result of increasing the health and well—being of the patient, is it successful? The 
answer to the surgeon may possibly be yes, but to the patient of a surgical procedure the 


answer is no. 


The Government is not required to accept a QMS_ until it demonstrates that it 
satisfactorily controls process and product in accordance with the Standard Inspection 
Clause. QMS demonstrations and audits may not discover faults in QMS until contract 
execution. Government auditors, while afforded protections outside the scope of ISO 


9000, must continue to be diligent in audit and surveillance procedures even under ISO 
9000:2000. 


Elaborate systems of checking to ensure procedural fidelity are administered but 
may not meet the results or desired outcome of the customer (41). Organizations that are 
realizing products must have the institutional ability and discipline to implement the 
underlying essence and concepts of the ISO 9000 Standard, not merely the “shall” and 
“will” of the document. In this area ISO 9000:2000 is addressing a major deficiency that 
has been a source of international criticism. ISO 9000:2000 yields private sector 
protections not previously provided, and makes ISO 9000:2000 congruent, in many 


aspects, with the protections of the Government’s Standard Inspection Clause. 


The ability to become a registered ISO organization and still produce scrap, 
especially under the ISO 9000:1994 version was evident industry-wide. ISO certification 
means only that the company’s affirmation is that it does what is says it does. The 
outcome could be good or bad, as seen in the hospital analogy. The ISO effort to address 


this shortfall is evident in the eight management principles of ISO 9000:2000, 
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particularly customer satisfaction, continual improvement, and process management 
approach. The eight- management principles and five “disciplines” of ISO 9000:2000 are 
too new to develop a statistically sound judgment if the principles and disciplines provide 
a higher-level of quality assurance. Often only one degree of separation exists between 
ISO 9000 registered companies, which consistently produce conforming, defect free 
material and those who are registered and do not. A respondent acknowledged: 

No registration process will guarantee product quality. The guarantee 

audit would be so expensive that it would be unaffordable, in addition, 

there are very few people that could perform an audit to the scope and 


detail required to verify every step in every process is acceptable. Even 
100 percent inspection is only 85% effective. 


Another respondent stated: ISO registration indicates that you have the 
minimum necessary business/management processes in place; it is the 
starting point for stellar quality not the zenith. 


The only absolute measure of conformance is 100% inspection and as was 
realized during the early stages of WW II, to conduct 100% inspection is cost prohibitive 
in manufacturing mass quantities of goods or technically difficult items to test, like 
software and modern weapon systems. Measures such as random sampling and statistical 
process control must be established as part of an organization’s metrics to measure 
product conformance. ISO 9000:2000 provides the requirement for product quality 
assurance but there are intangibles: warranty, consistency, quest for quality, process 
tools to continually improve, cost analysis, and product process quality issues and 


resolution (59). 


Another point to consider is that not all registrars or DCMA auditors are created 
equal, some are more conscientious than others, and each comes with his or her own 
personal biases and backgrounds influencing the audit (9). Just as not all of the operators 
of a piece of equipment in a factory are equally trained or equally proficient, neither are 
registrars and DCMA auditors. In the researcher’s view, quality assurance and 
management requires an integrated product and process development (IPPD) approach of 
teaming between industry, DCMA, and registrars to provide the greatest level of fidelity 
to build a quality product through the systems engineering approach and IPPD. The IPPD 
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and Systems Engineering approach fit well into the ISO quality management principles in 

addressing continuous process improvement and PQM. As one interviewee responded: 
[You need to] understand the essence of the processes and that is what 
Process Based Management helps you to do. The fact that everyone is 
looking at their processes through established metrics, and measuring the 


process against the metrics, you would be hard pressed to have a major 
systems failure. 


You can fool the auditors but if you use them as a tool to improve your 
process you can use that information to make real improvements with your 
company. People within the organization must embrace the need for their 
role in the quality process of the whole organization and its product. 


DCMaA plays a critical role in the policy versus perception of a quality system, or 
process versus the desired outcome of quality conforming product. DCMA’s 
involvement as contract administrator and evaluator of organizational quality systems 
during solicitation and during daily operations is of critical importance in the dynamic 
environment of quality assurance in DoD procurements. Experts agree no one standard 
or preaward survey can provide absolute guarantees of a 100% defect free product. 
Standards and surveys can and do provide valuable insight into an organizations ability to 


produce a quality product. 


The key to creating a quality product is to find the congruence between quality 
systems, manufacturing processes, and conforming product. Once congruence is found, 
then an organization must establish metrics to continually monitor the process and allow 
feedback at all levels to continually improve and address quality issues associated with 
not only the product but also the processes used to manufacture the product. When 
congruence is broken, the source of failure and the process that created the fault must be 
identified and explored. Corrective action taken then is distributed to all the stakeholders 
in the process that created the fault. The distribution of corrective action allows for 


continual process improvement of the organization. 


The goal of any quality policy is to address quality risk, as one major defense 


contractor summarized: 


Quality addresses risk. Well-defined, well-implemented quality systems 
greatly increase the percent of time you have no defects. 
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Varying opinions were found on what type of risk ISO 9000 addressed. Critical 
opinions of the change saw the ISO 9000 update, as a move to focus nore on supplier 
needs than those of the customer. Critics suggest that the ISO standard was “watered 
down” to allow registrars more latitude in ISO 9000 registration, thus increasing the 
marketability and profits of registrars of ISO and registrars customers. The elimination 
of the 20 elements in their opinion allows for broader and greater interpretation of the 


ISO standard. 


Opponents see a gradual trend among registrars favoring suppliers rather than 
buyers. Opponents believe the ISO focus is on producer risk rather than consumer risk. 
The focus creates an environment allowing registrars of ISO to certify more companies 
and thus increase the registrar market share and role in ISO. The researcher could find no 
evidence to support or deny this claim. However, to some degree this dilemma always 
occurs in manufacturing. Manufacturers balance producer versus consumer risk daily 
and manage the tradeoffs the two risks hold. Industry observations espousing ISO focus 


on producer risk followed these comments. 
According to one respondent: 


The buyer community has not been addressed in the 2000 version. The 
customers of the registrars influenced ISO 2000. I see it as a less effective 
document and aimed more towards the service industry rather than 
manufacturing. 


According to another: 


The new version is more concerned with producer risk than consumer risk. 
The ISO standard is neither good or bad, it is not a silver bullet of quality 
to an organization either. I see three new areas in ISO 2K, continuous 
process improvement, quality objectives, and customer satisfaction. While 
these are new to the document does it help be make better widgets? The 
whole focus of the three is nothing new [and] any good business is doing 
these things already or they would not be or continue to be in business. 
The document should be intuitive for any competitive market in a 
capitalistic society. ISO provides no edge it is generic [;] it wants 
everyone to be gray. 


Proponents of the change believe ISO better matches business movement to a 


process management approach. The same approach is used in state-of-the-art quality 
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assurance systems. The new standard also better resembles internationally recognized 
quality awards programs. The new standard creates a better framework from which an 


organization can build and define its own QMS. 


Often what organizations saw as either strengths or weaknesses of the previous 
versions of ISO 9000 was based on their own point of reference and ability to understand 
what some call the implied tasks of ISO 9000:1994. Interviewees saw the change as 
addressing many of the implied and not so well defined aspects of the 1994 version. The 
framework of ISO 9000:2000 is consistent with the total quality management concept and 
DoD 5000.2-R. The common goal in managing an organization is to create an 
environment so that it excels in all dimensions of products and services that are important 
to the customer (4). The following table will visually show the consistency between the 


three with each area outlining their respective tenets. 


ISO 9000:2000 DoD 5000.2-R 
Committed Involved Mgmt | Responsible Management Monitor Critical Processes 
An intrinsic ISO goal Customer Feedback 


Workforce empowered Internal Quality Audits / Mgt | Proactive not Reactive 
Principle (c.) Involvement 
of people 


Continuous Improvement Corrective Action & Process | Continuous Process 
Control / Mgt Principle (f) Improvement 
Continual improvement 


Partnering w/ Suppliers Intrinsic Control of Product | Root Cause Analysis & 
Goals Corrective Action Systems 


Statistical Techniques Capable Processes 





Table 2. TQM, ISO 9000, and DoD 5000.2-R Relationship, [After Refs. 8,26,47]. 


The respective tenets of TQM, ISO 9000, and DoD 5000.2-R serve a common 
purpose. They all support a better quality environment. DoD 5000.2-R is a DoD 
Regulation providing guidance and implementation of the DoD Acquisition System, 
specifically DoD 5000-2R (Interim), Part 5.2.3. ISO 9000:2000 is an international 
standard or measure to survey a system’s quality. TQM is a management philosophy 
providing tools and techniques for managers to follow for improved quality systems. 


(1,4,56) TQM is consistent with the Single Process Initiative set forth by Dr. Gansler in 
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his 1995 memorandum aligning best commercial practices to be better reflected in DoD 
acquisition and procurement activities (17). The quality management process shall be 


capable of the following key activities: (17,23) 


e Establish capable processes 

e Continuously improve processes 

e Monitor and control critical processes and product variation 

e Establish mechanisms for field product performance feedback 

e Implement an effective root-cause analysis and corrective action system 


Achievement of quality requires an effective quality management process in 
conjunction with effective business and technical practices. Achievement requires 
engineering and manufacturing practices that emphasize robust design along with 
enterprise-wide process maturity through continuous process improvement efforts. 
Benefits include first time pass quality, decreased cycle time, as well as reductions in 
rework, engineering changes, and inspections. These benefits translate into improved 
affordability and reduced production transition risk. (4,13,56) 


D. THE RELATIONSHIP BETWEEN DEFENSE CONTRACTORS AND 
DOD 


The change in quality assurance philosophy from MIL-Q-9858 to allowing 
contractors to use equivalent commercial quality assurance systems and ISO 9000 
standards has program and contract management implications. Some of these 
implications are discussed under the Advantages and Barriers section of this chapter. All 
those questioned agree the DoD SPI and use of the “most effective quality system” 
continues to allow a better quality environment. SPI and “most effective quality system” 


do not come without issue. 


The key issue is best explained by one of the respondents and addresses an 
overarching DoD problem facing industry and the DoD in instituting SPI and “most 


effective quality system.” 


The SPI of only one standard to industry is deceiving. MIL-Q and MIL-I 
were not canceled [sic]. They are not the preferred method according to 
DoD policy. So when we (DoD) stopped dictating what quality system to 
use to allow the “most effective quality system,” we opened the door for 
industry to choose the system. Many organizations decided to remain with 
the MIL-Q and MIL-I standards. DCMA now must not only understand 
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ISO 9000:2000 but all of the previous military standards still in use, many 
industry accepted quality standards like QS 9000 [that] the automobile 
industry uses, as well as ISO 9000:1994. The way the FAR is written we 
are discouraged to dictate to an organization any of these systems as long 
as they meet the “most effective” criteria. The challenge of understanding 
all of these systems, although there are many similarities between them, is 
difficult for DCMA to consistently and to appropriately manage across the 
numerous sectors of business DoD is contracted with. 


DoD suppliers’ complaint about the cost of having two or more systems 

prior to SPI is only a half-truth. Most organizations only used one system, 

MIL-Q; the cost of quality is something an organization will commit to 

even if it is not contractually outlined. The cost of implementing ISO, if 

they chose to leave MIL-Q, is only a 5% to 10% increase. The differences 

between ISO 9000-1994 and MIL-Q 9858 were not significant enough, in 

my opinion, to warrant significantly more cost. Suppliers wanted DoD 

inspectors out of their factories and in SPI industry saw a way to 

accomplish the elimination of in-house DCMA representatives. Industry 

and NATO’s push to use ISO 9000:1994 instead of the Allied Quality 

Assurance Procedures (AQAPs) (see Appendix C) pushed DoD to make 

the change to allow QMS like ISO. 

One proposed solution to address the numerous QMS that DoD organizations 
such as DCMA must be versed in is to mandate the use of ISO 9000. All respondents 
agreed that ISO should not be imposed. The reason stated predominately was the burden 
it would place on small businesses. Small business may choose to implement ISO. The 
primary burden is the financial and resource cost of third party registration (16). DoD 
does not mandate third party registration, only the existence of an adequate QMS. The 
intent of ISO 9000, like other quality systems, is to add value to the organization through 
enhanced quality and process control. The expectation is once a system like ISO 9000 is 


implemented, organizational effectiveness and efficiency resulting in better processes and 


products absorb implementation costs. 


Assume for discussion that DoD mandates use of ISO 9000, and more specifically 
requires ISO 9000 with third party registration. A DoD ISO 9000 registration mandate 
could be very problematic and could result in a breakdown of the arms-length 
relationship between DoD, registrars of ISO, and contractors. The view is based on the 
underlying issues raised in many of the respondents’ comments but one of the 


interviewees described it best: 
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If DoD chooses to mandate ISO, the only way to make it work would be to 

select one registrar to handle all DoD contracts. This would be required to 

ensure a level playing field among DoD contractors. 

There is also a problem with picking only one registrar. Registrars become 
financially tied to the successes and failures of the companies they evaluate. The success 
of the registrar’s business is tied to registration, and registrars only get return business if 
they register an organization. Independent registrars are not, in fact, independent. To 
ensure this conflict of interest would not occur, DoD would need to pay the selected 
registrar for all DoD business, instead of having the private organization or contractor 
assume the cost of implementing a quality system. Currently, DoD can benefit from a 
company’s quality system without absorbing all of the cost for the implementation of that 
system. The cost of the periodic and third year audits would be too high for DoD to 
absorb and the conflict of interest issue is not easily resolved. In one quality manager’s 
opinion: 

DoD cannot use the blanket policy of choosing only ISO registered 

organizations to do business with. Commercial buyers and producers 

must be better than that (ISO). Selection must be process and skill 

determined and situation dependent. You want to use source inspection 

and performance data and feed that into a process of source selection. 

DCMA and DoD have not done a good job of doing that they have not 

invented the proper tools to make that process work as well as industry 

has. A good example of how this works is a vendor-rating program. 

Product is inspected; performance data is generated and processed into our 

vendor-rating program across the organization. We use it as a tool for 

future supplier selection. [see DCMA request the data but I don’t see a 

documented transfer or process to provide the information across DoD 

organizations. 

Under the current regulation, the DoD reaps the rewards of industry applying their 
own QMS without burdening that cost on Government contracts. DCMA’s responsibility 


is to assure the system best suites the needs of the customer and that the overhead rates 


properly reflect industry cost. 


The DoD acquisition workforce must currently be versed in numerous 
commercial and military based quality management systems. Education and training 
cannot only focus on quality standards such as ISO 9000 and ANSI 9000 series, but on 


industry and market specific quality systems as well, including military QMS. The 
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education and training of the DoD acquisition workforce is more critical under SPI. 


Specifically, ISO 9000:2000 training is currently lacking. 


The DCMA organizational view is that ISO has not undergone a significant 
change. Accordingly, the agency is not sponsoring training on the change, beyond 
conducting an internal review and dissemination of the ISO 9000:2000 Standard. DCMA 
is encouraging local commands to send QMS lead auditors to training, to conduct “train 
the trainer’ instruction. Once lead auditors are trained they will train the local 
organizations on ISO 9000:2000 changes. The problem associated with this strategy is 
that DCMA field activities work under limited budgets. If DCMA as an organization 
does not recognize the importance of funding training from a corporate level, then what 
emphases will local DCMA Commanders place on their employees receiving adequate 
training equivalent to what their industry counterparts are receiving? The lack of DCMA 
funding to address this issue may result in inconsistent and inadequate audits of industry 
using ISO 9000:2000 as their QMS. One DoD employee surmised. 

The DCMA Headquarters view is [ISO 9000] 2000 is not significantly 

different from [ISO 9000:] 1994. The civilian sector sees this [ISO 

9000:2000] differently and is committing resources to make the change. 

DCMA should follow suite, at least partially, to assure our auditors are 


able to perform an audit at an appropriate level of knowledge and 
expertise. 


Defense industry quality assurance managers interviewed believe that more 
resources should be placed in training DCMA representatives on ISO 9000:2000. 
DCMA’s ability to adequately audit and provide surveillance for the DoD customer is 
directly correlated to product quality and performance, which results in long term cost 
savings over the life of the product or service. A few well-spent dollars for training may 
result in saving resources and higher quality products for DoD users resulting in 


decreased cost and greater reliability over a system’s life cycle. 


Training and education is more difficult with numerous QMS for the workforce to 
become familiar with. Previously MIL-Q-9858 and MIL-I-45208 provided detailed and 
prescriptive guidance in the form of two handbooks, commonly referred to as H50 and 
H51 (38,39). The handbooks provided cases and examples for the auditor of MIL-Q- 


9858 to use when conducting audit or when approached with varying audit and 
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surveillance circumstances. No such handbook exists for DCMA to use when applying 
the numerous audit functions it is asked to perform of military and commercial QMS. 
Handbooks 50 and 51 are used with MIL-Q-9858 and MIL-I-45208, respectively. The 
need for case studies and audit surveillance training and experience remains. In the thesis 
writer’s research, no library of consolidated lessons learned is available across DoD or 
DCMaA to serve and to share the daily wealth of knowledge and experience gained by the 
acquisition workforce in administering and managing QMS. An interactive and integrated 
approach incorporating best practices and lessons learned would aid DoD and industry in 


establish a consistent approach to DCMA audit and surveillance. 


Training and education are central to cultural change. The introduction of SPI 
and the “most effective quality system” is slowly changing the cultural relationship of the 
DoD and defense industry. DCMA culture, in many aspects, is no different than the 
culture in many other public or private organizations. Culture is a most difficult concept 
to understand and is often difficult to change. Without the ability to change, organizations 
in the private sector lose their competitiveness; in the public sector, organizations that 


will not change soon lose their relevance. 


In the public arena measures of success and performance are often ambiguous or 
conflicting, with no one person controlling the agenda of the organization, and DCMA is 
no exception. This is not to say organizational change cannot occur. The process of 
cultural change in public organizations is different in that leaders of the organizations 
cannot provide rewards for cultural change without budgetary authority, or alternatively 
laws and regulations may dictate how rewards are given to public organization members. 
Without the ability to match cultural change to some type of incentive or reward for 
changing, a catalyst does not exist to make the organization’s stakeholders adopt any 
significant change. Leaders of public organizations often must be much more creative in 
incentivizing change, for without incentive an organization’s culture will remain the 
same. One DCMA quality manager spoke about the DCMA view toward ISO: 

In 1988 DoD wanted nothing to do with ISO 9000. The culture, while 

slowly changing, still has yet to wholly embrace the SPI and ISO 9000. 

Culture is difficult to change. The aging of our workforce, the decline in 

our quality assurance workforce numbers from 24,000 in 1995 to 12,000 


currently [numbers are approximate] and our inability to attract some of 
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the best talent in the field of quality assurance and management are factors 
affecting the DCMA culture. 


The dominant view is that DCMA culture has changed, but still has some areas to 
improve. Of greatest concern to industry was the “graying” DCMA workforce trained 
under the detail specifications and adversarial relationships between DoD and industry of 
the past. The difficulty of DCMA to attract entry-level and apprentice employees to its 
workforce is a Government-wide issue. Positions within the Federal Government are an 
unattractive alternative to careers in the private sector, given the high employment rates 
of the 1990s. Federal workforce reductions, particularly those in the DoD, eliminated 
many of the junior DoD employees through Federal protection of senior DoD workforce 
members, causing a potentially severe problem in the ability of the DoD civilian 
workforce to address future missions. One defense industry observer said, 

...DCMA reduced their personnel by two-thirds over the past decade [and] 

all you had left was older, more senior employees. These employees come 

from a past culture of inspection and monitoring, [reinforcing the position] 

that we (DCMA) cannot trust the contractor. You reach a point where it is 

very difficult for DCMA to change the culture. It is a generational issue. 

They (DCMA) are being taught, trained, and told to do it differently but 

can’t or won’t change due to the past cultural influence. The environment 


has become increasingly more technical with change occurring more 
rapidly than in the past. 


According to some DoD officials industry pushed for the issuance of the Single 
Process Initiative (SPI) in hopes to reduce and or eliminate the amount of DCMA 
oversight and surveillance (23,59). What SPI failed to eliminate for industry, severe 
DCMA budgetary and resource constraints did. DCMA oversight and surveillance little 
resembles what it was in the 1980s. SPI and resource constraints have forced DCMA to 
refocus its audit and surveillance techniques from oversight to insight. The researcher 
does not see this as a weakness of DCMA but rather a challenge to continue to ensure 
products conform to the contract performance specifications within the limits of public 


funds. 


Some industry respondents argue that DCMA should reduce oversight with ISO 
registered companies. The best approach seems to be based on various factors, and third- 


party ISO registration is one. DCMA must base the amount of oversight and audit it 
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conducts on accurate estimates of the risk associated with the contract and DCMA’s 
quality assurance role should focus on this area. DCMA cannot become detached from 
the customer’s product requirements. The DCMA workforce needs a clear understanding 
of customer requirements due to their contract oversight responsibilities. DCMA ACOs 
and QARs are the first and often only representation the user has on the production floor. 
Here lies significant risk to product conformity if DCMA fails to understand and properly 


recognize customer requirements. 


Under constrained resources the DoD should make use of its risk management 
system to evaluate what level of contract oversight is appropriate. DoD organizations 
like DCMA should consider internal organizational competencies and strengths as part of 
their risk management evaluation of defense contractors. Many defense industry quality 


managers observed the following excerpt: 


The registrars are head-and-shoulders above DCMA on their ability to 
evaluate compliance to the ISO Standard. They (ISO) are much more 
capable of understanding the systems evaluation of an organization and 
the organization’s written procedures to match the system. Registrars look 
at the entire business system. Registrars are not very good at looking at a 
product and measuring its conformity and compliance to the procedures. 
DCMA is hardware oriented and much more familiar with the hardware 
requirements. 


And a similar observation: 


There are differences among registrars and differences among DCMA 
auditors. Government representatives come from different agencies, from 
various areas and communities, with varying objectives against the same 
contract. DoD agencies waste time with the infighting that occurs 
between the various DoD stakeholders. It constrains DCMA and all the 
other DoD agencies to adequately evaluate an organization because DoD 
cannot reach consensus on what they expect of the contractor performance 
against the contract. The roles of contract administration are not agreed 
upon between DCMA and the buyer. 


The PCO and ACO must clearly designate and define the areas of contract 
administration and procurement functions. The resident DCMA 
representatives should take the lead in deciding the jurisdiction of the PCO 
and the ACO in many of these matters. 


The DoD must provide clear guidance to DCMA as to what and how they are to 


be used as a tool to mitigate contract risk. DCMA is not resourced financially or, in 
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manpower (adequately trained and in numbers) to properly address all of the missions it 
is asked to perform. DoD users of DCMA must insure the scope of audits and 
surveillance required is clearly defined. This helps ensure resources are not wasted on 
nonvalue added tasks. Third party ISO registration is just one area, of many, that 
DCMA must evaluate and the extent of confidence that DCMA should place on the 


registration. 


There is not a cookie cutter answer to the right quality assurance program; 
approaches vary from industry to industry and within industry sectors. ISO 9000 is not a 
silver bullet and once shot or put into place, it does not necessarily mean output will be 
flawless. Oversight and audit personnel must then build a broad base of knowledge, 
experience, and efficiency in performing and carrying out oversight and audit functions. 
Auditors and quality assurance specialists must go to the shop or factory floor and speak 
to the employees that actually put quality into the product through their own skill, 


knowledge, and pride in the outcome of their respective manufactured item or service. 


Market and industry analysis and research is key to making a proper assessment 
of a quality processes existence in whatever area the auditor or contracting officer is 
working. Armed with this knowledge, oversight and audit functions can carry out their 
analysis effectively resulting in little interference creating an environment where 
Government auditor and contractor team develop and produce the best value outcomes 
for both industry and Government. System managers, contracting officers as well as 
auditors of quality management must make a quality assurance determination. The costs 
and benefits of a contractor’ s quality management system must be evaluated to determine 
that the quality system that will produce the desired outcome sought by the buyer. This 


determination must be made on a contract-by-contract basis. (3,9,21) 


Some observers advocate that successful completion of ISO registration should be 
equivalent to a satisfactory preaward survey. Numerous problems are associated with this 
assertion, which brings to light many of the weaknesses of the ISO 9000 certification. 
Proponents argue ISO auditors use many survey procedures that are almost exactly the 
same as are used in preaward surveys. While to some extent this is true, the ISO audit for 


registration occurs once, with cursory audits every 6 months performed by ISO 
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registration teams. Complete reviews are carried out only after three years of being 
registered as ISO 9000 compliant quality organizations (9,53). This frequency and depth 
may not be enough to ensure a quality system meeting the needs of a particular 


procurement. 


As was mentioned earlier, every quality system must be evaluated on a case-by- 
case basis as it applies to the given procurement circumstance. For example, assume that 
the US Army has contracted with a company to manufacture a replacement for its aging 
medium truck fleet. One of the contractors in the competitive range is an ISO 9000 
certified company but has not produced a truck of this type and series since World War 
Il. It would be advisable, and appropriate within the Standard Inspection Clause, to 
review the company’s current quality system. Concerns throughout industry mention that 
registered auditors supposedly qualified by the ISO fail to provide adequate guidance and 


measures in certifying organizations as ISO 9000 compliant. (3,14,37,53). 


Who then will be able to distinguish the difference between good or poor ISO 
9000 auditors if not through the Government’s own oversight and auditing groups? FAR 
Part 46.202 prescribes that the Government “bears the responsibility of determining 
whether the standard is implemented properly and adequately,’ not the ISO. One 
interviewee whose views were representative of the majority opinion on DCMA 
oversight offered: 

The amount of surveillance or oversight should not be based on any 

redundancies found between registrars and DCMA but driven more by 

how well the process is working and measuring up to the oversight DCMA 

feels they need to conduct and as to how we are evaluated to contractual 

performance measures. The relationship, if one exists, is if ISO 9000 has 

helped us to improve and organize our process to a point where DCMA is 

more confident, not the ISO registration. It is a matter of confidence and 

risk associated with the contract, not a correlation of registrars and 

DCMA. 

The Government would assume significant risk in delegating audit and oversight 
authority to any outside organization. The Government risks losing the opportunity to 
guide and mentor contractors through the acquisition cycle and its association with the 


respective contractor. The Government would also risk losing valuable insight required 


to maintain the capability for monitoring and establishing a base from which future 
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quality assurance auditing must occur. ISO 9000 certification should be promoted and is 
encouraged by DoD. While ISO 9000 is no guarantee of a quality process or product it 
provides a certain level of confidence towards meeting requirements in quality assurance 


the Government might expect inherent to a contractor’s quality system process. 


Publicity concerning low quality can ruin the reputation of a company and can 
have the same effect on any DoD program where public trust is involved and expected in 
any procurement (3,51). DoD has worked very hard over the past decade to reverse a 
loss of public trust associated with some procurement activities in the 1980’s (23,44). 
DoD must be aware that suppliers that are QS 9000 certified, such as Firestone, may pass 
the surveillance audit, but the customer is still not guaranteed protection from defects (9). 
In the public sector, the Government is held accountable for the actions of its suppliers. 
Only when the contractor and Government take proactive efforts can a truly effective and 


efficient quality management system work. 


61 


THIS PAGE INTENTIONALLY LEFT BLANK 


62 


Vv. CONCLUSIONS AND RECOMMENDATIONS 


A. INTRODUCTION 

ISO 9000: 2000 assimilates the 20 elements of ISO 9001:1994. The eight quality 
management principles and five disciplines of ISO 9000:2000 incorporate the essence of 
the 20 elements. ISO intent is for the 2000 version to address shortcomings of the 1994 
version and to update ISO to current quality management and assurance practices and 


techniques. 


A lack of understanding of the limitations and capabilities of ISO 9000 exists 
within the DoD acquisition workforce and defense contractors. Neither ISO 9000:1994 
nor ISO 9000:2000 guarantees high quality product if the QMS system is not tailored to 
the product or the workforce fails to follow the processes and practices of the QMS. The 
ISO 9000:2000 focus on management involvement, process quality management, 
continual improvement and customer satisfaction recognizes the essentiality of 
management involvement. ISO 9000:2000 provides guidelines for building an 
environment that encourages and supports the process outcomes desired by the customer. 
Whereas ISO 9000:2000 is no silver bullet, it will help to achieve a clear understanding 
of customer requirements, emphasis on quality from the front-line workforce to corporate 
executives, and proven procedures monitored by appropriate metrics which allow for 
contractual improvement arm an organization with the tools to consistently produce 


conforming products and thus higher customer satisfaction. 


ISO 9000:2000 can provide the foundation and framework for a quality-achieving 
organization but without the materials (the policies, the techniques, the procedures, and 
the metrics) discussed in Chapter’s H-IV, all that remains is a frame of a house, an empty 


shell, never realizing its QMS potential. 


The role of DCMA, and other DoD contract management activities, is critical in 
an environment of misplaced trust and misunderstood capabilities. DCMA is the DoD 
ombudsman, the frontline worker, and the advocate who provides QA oversight and 
inspection of products destined for the Department of Defense. Quality assurance 


specialists assigned to DCMA have the key role as the Government’s quality assurance 
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representative who approves the quality management system for use in producing a 


particular warfighting system. Application of ISO 9000:2000 must meet their standards. 
B. CONCLUSIONS 


The list that follows is a consolidation of the conclusions resulting from this thesis 


research. 


DCMA role is more complex under the SPI. Under the SPI DCMA is 
responsible to evaluate numerous QMS. Prior to 1996, with few 
exceptions, it was only the MIL-Q- Standards. 


ISO provides a baseline or framework for a QMS. Culture, manufacturing 
ability, technological know-how, leadership, management emphasis, and 
process quality control and management metrics provide the highest level 
of confidence and reduction of risk associated with a procurement. 
However, ISO registration does not guarantee to the DoD that defense 
contractors will manufacture a conforming product or even understand the 
customer’s requirement through the contract and SOW. 


ISO 9000:2000 contains the necessary elements of a quality management 
system that, if correctly applied, is suitable for higher-level quality 
requirements. In every case the QMS must be examined against the 
quality requirements of the product itself. This is accomplished in 
accordance with the provisions of the Standard Inspection Clauses used in 
all Government contracts. 


ISO 9001:2000 principles and disciplines integrate the 20 elements of ISO 
9001:1994. The integration reduces a stovepipe approach to QMS 
implementation and _ utilization. The ISO 9000:2000 takes a more 
proactive approach than ISO 9000:1994, considering as organization’s 
culture and environment, leadership attention, and customer. 


C. RECOMMENDATIONS 


The list that follows is a consolidation of the recommendations based on the 


conclusions and information gathered from this thesis research. 


The Department of Defense should use ISO 9001:2000 as its benchmark 
Quality Management System for use when a higher-level quality is 
required. 


The Defense Acquisition University and applicable service schools should 
teach ISO 9000:2000 as the quality management system baseline. 
Additionally, these schools should describe the application of ISO 
9000:2000 in DoD contracts. 


Buying activities should work closely with DCMA to ensure a systematic 
approach for contractual implementation of ISO 9001:2000 
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DCMA should continue its current practice of issuing a Statement of 
Qualification to contractors found in compliance with higher-level 
contract quality system standards, e.g., ISO 9001:2000. 


Interpretation of ISO 9001:2000 language should also be addressed 
through internal DCMA guidance and DAU training. 


DCMaA in conjunction with DAU schools should establish a virtual library 
for use DoD-wide, capable of providing the instructional tools that 
Handbooks 50 and 51 gave the acquisition workforce under MIL-Q-9858 
and MIL-Q-45208. 


DCMA should maintain a library, which contains a compilation of defense 
contractors who have used ISO 9000:2000, their Registrars, contractors’ 
history of QMS performance, and lessons learned. This resource would 
provide current and future acquisition employees information upon which 
to base future ISO 9000 QMS decisions. 


USD (AT&L) should establish a policy that addresses current and future 
QMS updates specifically focusing on contractual aspects of QMS 
updates. 


DCMA should continue to update its QMS checklist to reflect changes 
implemented by ISO 9000:2000. 
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APPENDIX A. LIST OF COMPANIES PARTICIPATING IN 
INTERVIEWS 


AM General 
Boeing 

General Dynamics 
Honeywell 
Lockheed Martin 
Motorola 
Northrop-Grumman 
Oshkosh 

Raytheon 

Stewart and Stevenson 
TRW 

United Defense 
DCMA-East 


DCMA-HQ 
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APPENDIX B. ISO TO MIL-Q CROSS REFERENCE 








Table 3. MIL-Q-9858 to ISO 9000:1994 Family, [After Ref. 3]. 
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ISO/FDIS 9001:2000 ISO 9001:1994 
1 Scope 1 
1.1 General 
1.2 Application 
2 Normative references 2 
3 Terms and definitions 3 
4 Quality management system [title only] 
4.1 General requirements 4.2.1 
4.2 Documentation requirements [title only] 
4.2.1 General 4.2.14 4.5.1 


4.2.2 Quality manual 4.2.1 

4.2.3 Control of documents 4.5.14+45.2+4.5.3 
4.2.4 Control of quality records 4.16 

5 Management responsibility [title only] 

5.1 Management commitment 4.144.1.2.2+4.2.1 
5.2 Customer focus 4.3.2 

5.3 Quality policy 4.1.1 

5.4 Planning [title only] 

5.4.1 Quality objectives 4.1.14 4.2.1 

5.4.2 Quality management system planning 4.2.3 

5.5 Responsibility, authority and communication [title only] 

5.5.1 Responsibility and authority 4.1.2.1 

5.5.2 Management representative 4.1.2.3 

5.5.3 Internal communication 

5.6 Management review [title only] 4.1.3 

5.6.1 General 4.1.3 
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ISO/FDIS 9001:2000 ISO 9001:1994 
5.6.2 Review input 4.1.3 
5.6.3 Review output 4.1.3 
6 Resource management [title only] 
6.1 Provision of resources 4.1.2.2 
6.2 Human resources [title only] 
6.2.1 General 4.1.2.24+4.2.3+4.18 
6.2.2 Competence, awareness and training 4.18 


6.3 Infrastructure 4.1.2.2 + 4.9 
6.4 Work environment 4.9 


7 Product realization [title only] 


7.1 Planning of product realization 4.2.34+4.9+4.10.1 


7.2 Customer-related processes [title only] 


7.2.1 Determination of requirements related to the product | 4.3.2 + 4.4.4 
7.2.2 Review of requirements related to the product 4.3.2+43.34+43.4 


7.2.3 Customer communication 4.3.2 


7.3 Design and development [title only] 





7.3.1 Design and development planning 4.4.24+443+4+44.6 


+ 4.4.74+4.4.8 
7.3.2 Design and development inputs 
7.3.3 Design and development outputs 


7.3.4 Design and development review 

7.3.5 Design and development verification 

7.3.6 Design and development validation 

7.3.7 Control of design and development changes 
sopIss00120 
Srapuchasngtiteony) SS 


ISO/FDIS 9001:2000 


ISO 9001:1994 


7.4 Purchasing [title only] 
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7.4.1 Purchasing process 4.6.2 
7.4.2 Purchasing information 4.6.3 








7.4.3 Verification of purchased product 4.6.4 + 4.10.2 + 
4.10.3 + 4.10.4 

7.5 Production and service provision [title only] 

7.5.1 Control of production and service provision 4.9 + 4.10.3 + 4.15.6 
+ 4.19 

7.5.2 Validation of processes for production and service 4.9 

provision 


7.5.3 Identification and traceability 4.844.105 + 4.12 
7.5.4 Customer property 4.7 


7.5.5 Preservation of product 4.15.24+4.15.3 + 
4.15.4 +.4.15.5 + 
4.15.6 


7.6 Control of monitoring and measuring devices 4.11.14+4.11.2 





8 Measurement, analysis and improvement [title only] 


8.1 General 4.10 + 4.17 + 4.20.1 


8.2 Monitoring and measurement [title only] 


8.2.1 Customer satisfaction 
8.2.2 Internal audit 4.17 


8.2.3 Monitoring and measurement of processes 4.94417 + 4.20.1 





8.2.4 Monitoring and measurement of product 4.10.2 + 4.10.3 + 
4.10.44 4.10.5 + 
4.20.1 

8.3 Control of nonconforming product 4.13.1 +4.13.2 





8.4 Analysis of data 4.14.24+414.3 4 
4.20 


8.5 Imp rovement [title only] 
8.5.1 Continual improvement 4.1.3 
8.5.2 Corrective action 4.14.1 4+ 4.14.2 


pes: 


8.5.3 Preventive action 4.14.1+4.14.3 


Table 4. ISO 9001:2000 to ISO 9001:1994, [From Ref. 55]. 
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APPENDIX C. NATO AQAP 


NATO Quality System Standards (AQAP’s) 


NATO has established a set of standardized procedures in an effort to support 
quality acquisition activities between North Atlantic Treaty Organization (NATO) 
member nations. This approach ensures each NATO nation performs consistent quality 
assurance activities. The two NATO Standardization Agreements (STANAGs) that 
govern NATO quality assurance activity are the STANAG 4107 and STANAG 4108. 


The STANAG 4107, titled Mutual Acceptance of Government Quality Assurance, 
sets forth the terms and conditions under which mutual government quality assurance of 
military material and services is to be performed by the national authority of one NATO 
country on request of another NATO country or NATO organization. The STANAG 
4108, titled Allied Quality Assurance Publications (AQAP’s), concerns the standardized 
development, updating and application of the AQAPs in the procurement of NATO 
defense material. 


The original AQAP’s established for quality assurances were numbered | through 
15. These AQAP’s were similar to documents used by the U.S. DoD for quality 
assurance activity. AQAP-1 was similar to MIL-Q-9858A, AQAP-2 was similar to MIL- 
HDBK-50, AQAP-4 was similar to MIL-I-45208A, etc. 


In the early 1990’s, NATO recognized that quality was becoming a major factor 
in international trade and that the ISO standards offered harmonization of NATO quality 
concepts and terminology with the European Community and other countries world-wide. 
NATO decided to revise the existing AQAP standards in order to embrace these new 
international quality standards. 


In February 1993, NATO canceled the AQAP 1-15 documents and adopted a new 
series of AQAP’s that incorporate the ISO 9000 series quality system standards. 


SUPERSEDED OR CANCELED AQAP’s 


AQAP NUMBER DISPOSITION 
AQAP- 1 Superseded by AQAP-110 
AQAP-2 Superseded by AQAP-119 
AQAP-3 Canceled 

AQAP-4 Superseded by AQAP- 130 
AQAP-5 Superseded by AQAP-119 
AQAP-6 Superseded by ISO 10012-1 
AQAP-7 Superseded by ISO 10012-1 
AQAP-8 Transferred to AC/301 
AQAP-9 Superseded by AQAP- 131 


15 


AQAP- 11 Transferred to AC/301 


AQAP- 13 Superseded by AQAP- 150 
AQAP- 14 Superseded by AQAP- 159 
AQAP- 15 Superseded by ISO 8402 


This new series of standards is sometimes referred to as the “Century Series” 
because of its numbering system. 


AQAP- 100 - General Guidance on NATO QA 

AQAP- 110 - NATO QA Requirements for Design, Development, and Production 

AQAP- 119 - NATO Guide to AQAPs - 110, - 120 and -130 

AQAP- 120 - NATO QA Requirements for Production 

AQAP- 131 - NATO QA Requirements for Final Inspection 

AQAP- 150 - NATO Quality Assurance Requirements for Software Development 

AQAP- 159 - NATO Guidance for the Use of AQAP- 150 

AQAP- 160 - NATO Integrated quality requirements for software throughout the 
life cycle. 

AQAP- 169 - NATO Guidance for the Use of AQAP- 160 

AQAP- 170 - NATO Guide for the Delegation of Government Quality Assurance 

STANAG 4107 - Mutual Acceptance of Government Quality Assurance and 
Usage of the Allied Quality Assurance Publications 

AC/250 -HNBK - The Group of National Directors for Quality Assurance 


Identified below are the NATO Contractual AQAP’s that directly reference the 
ISO 9000 contractual standards: 


AQAP 110: NATO Quality Assurance Requirements for Design, Development, 
and Production directly references the elements contained in the ISO 9001 standard. 


AQAP 120: NATO Quality Assurance Requirements for Production directly 
references the elements contained in the ISO 9002 standard. 


AQAP 130: NATO Quality Assurance Requirements for Inspection directly 
references the elements contained in the ISO 9003 standard. 


It should be noted that there are differences between the AQAP’s and ISO 9000 
quality system standards. The AQAP standards contain additional requirements 
(supplements) to certain ISO elements. These supplements provide additional direction 
and guidance on NATO quality system needs. 


Governments and organizations sometimes place contracts with contractors 
located outside their own country. Contract administrative oversight and support for those 
contracts may be available via established International Agreements and/or 
Memorandums of Understanding (MOUs) with the various nations and organizations. 
Contract administrative support may also be available through the Defense Contract 
Management Agency International (DCMAI) which has support and oversight 
capabilities in many countries 
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APPENDIX D. ISO FAMILY 


The ISO 9000 Family 


The standards, guidelines and technical reports which make up the ISO 9000 family and 
which are listed below are available separately, or as collections. The ISO 9000 
Compendium presents the ISO 9000 family in hard copy form. 


Standards and guidelines 


ISO 9000:2000, Quality management 
systems - Fundamentals and 
vocabulary 


ISO 9001:2000, Quality management 
systems - Requirements 


ISO 9004:2000, Quality management 
systems - Guidelines for performance 
improvements 


ISO 19011, Guidelines on Quality 
and/or Environmental Management 
Systems Auditing (currently under 
development) 


ISO 10005:1995, Quality 
management - Guidelines for quality 
plans 


ISO 10006: 1997, Quality 
management - Guidelines to quality 
in project management 


ISO 10007:1995, Quality 
management - Guidelines for 
configuration management 


ISO/DIS 10012, Quality assurance 


requirements for measuring 


Purpose 


Establishes a starting point for understanding the 
standards and defines the fundamental terms and 
definitions used in the ISO 9000 family which 
you need to avoid misunderstandings in their use. 


This is the requirement standard you use to assess 
your ability to meet customer and applicable 
regulatory requirements and thereby address 
customer satisfaction. 


It is now the only standard in the ISO 9000 family 
against which third-party certification can be 
carried. 


This guideline standard provides guidance for 
continual improvement of your quality 
management system to benefit all parties through 
sustained customer satisfaction. 


Provides you with guidelines for verifying the 
system's ability to achieve defined quality 
objectives. You can use this standard internally or 
for auditing your suppliers. 


Provides guidelines to assist in the preparation, 
review, acceptance and revision of quality plans. 


Guidelines to help you ensure the quality of both 
the project processes and the project products. 


Gives you guidelines to ensure that a complex 
product continues to function when components 
are changed individually. 


Give you guidelines on the main features of a 
calibration svstem to ensure that measurements 


a 


Standards and guidelines 


equipment - Part 1: Metrological 
confirmation system for measuring 
equipment 


ISO 10012-2:1997, Quality 
assurance for measuring equipment - 
Part 2: Guidelines for control of 
measurement of processes 


ISO 10013:1995, Guidelines for 
developing quality manuals 


ISO/TR 10014:1998, Guidelines for 
managing the economics of quality 


ISO 10015:1999, Quality 
management - Guidelines for training 


ISO/TS 16949:1999, Quality systems 
- Automotive suppliers - Particular 


requirements for the application of 
ISO 9001:1994 


Table 5. 


Purpose 


are made with the intended accuracy. 


Provides supplementary guidance on_ the 
application of statistical process control when this 
is appropriate for achieving the objectives of Part 
1. 


Provides guidelines for the development, and 
maintenance of quality manuals, tailored to your 
specific needs. 


Provides guidance on how to achieve economic 


benefits from the application of quality 
management. 
Provides guidance on the development, 


implementation, maintenance and improvement 
of strategies and systems for training that affects 
the quality of products. 


Sector specific guidance to the application of ISO 
9001 in the automotive industry. 


ISO 9000:2000 Family, [From Ref. 25]. 


The US Equivalents to the ISO Family 
ISO 9000 Is The Same As ANSI/ISO/ASQ Q9000 
ISO9000 = ANSI/ISO/ASQ Q9000 
ISO9001 = ANSI/ISO/ASQ Q9001 
1SO09004 = ANSI/ISO/ASQ Q9004 
ISO9000 is the international reference 
ANSI/ISO/ASQ 9000 is the U.S. reference (39) 
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APPENDIX E. EXECUTIVE AND DOD POLICY GUIDANCE 


e Office of Management and Budget Circular A-76 


oO 


Federal policy implementing Federal Activities Inventory Reform 
Act 


Authorizing the use of nongovernmental standards and 
specifications in contracts and the procurement of “off the shelf” 
(commercial) items for DoD use 


Applicable to the majority of executive agencies Government 
should not compete with its citizens Government shouldn’t carry 
on any activity if product or service can be procured more 
economically from commercial sources 


e USOD Deutch Policy Letter Dated 14 Feb 94 


oO 


Program offices are authorized to use ISO ANSI/ASQC Q9000 
Quality Series standards as follows: 


° In contracts for new programs 

e In follow-on efforts 

e In current contracts on a case-by-case basis 

e Third party registration NOT required 

e Third party registration NOT a substitute for Government 
oversight 

e Commercial level quality systems will be recognized if able 


to satisfy Government’s needs 


e USD Perry Policy Letter Dated 29 June 94 


oO 


Redefines order of precedence for specs and _ standards, 
performance specs, commercial Specs 


MILSPECS and MILSTDS (only with waiver) 
Change in DFAR 


All MILSPECS and MILSTDS related to management and 
manufacturing will be phased out suppliers encouraged to use 
commercial standards 


Reduce government oversight 


e SECDEF Memorandum May 10, 1995 


oO 


Subject: Use of Integrated Product and Process Development and 
Integrated Product Teams in DoD Acquisition. 
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e Quality products are best achieved through integrated 
developments of the product and _ its associated 
manufacturing and support processes, through a systems 
engineering approach. 


SECDEF Memorandum Dec. 6, 1995 


e) Subject: Common Systems / ISO-9000 / Expedited Block 


e DoD policy to allow use of single processes in a 
contractor’s facility. Major DoD initiative to allow industry 
to be more efficient, improve quality and reduce overall 
cost of acquiring products 


USD (A&T) Memorandum Sept. 18, 1997 


fe) Subject: “Requiring Processes on Contract” 
e DoD should not require processes in solicitations and 
contracts 
e DoD should rely on performance based requirements 
whenever practicable 


Principal Deputy Under Secretary of Defense (PDUSOD) Policy Mr. Dave 
Oliver letter March 19, 2001 


0) Subject: ISO 9001:2000 Quality Management Systems, Transition 


° Monitor contractor transition to new standard 


DCMA Policy from DCMA “One Book Supplier Quality Assurance Chapter 4.4 


MG Drews letter dated Nov 18, 1996. 


o) DCMA must assure organizations quality system complies with 
contract requirements and use existing data for evaluating 
compliance of organizations quality system 


oO DCMA must audit when: 


e existing data unavailable or inadequate 
e directed by customer 
e organizations performance indicates 
e system not in compliance with contract 
e requirements 
e quality system has been substantially 
e changed 
oO Evaluation of Quality System two basic methods: 
e Evaluation by assigned specialists 
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e Formal system audit 
Quality System Evaluation will be performed on the quality system 
cited in the contract 


DCMA Personnel Qualifications: 


e DAWIA Level II 

e Manufacturing and QA career field, DCMA auditors meet 
ISO 10011-2 requirements 

Quality System Audit 


e Tailored to reflect scope of contract requirements 

° Records to indicate how confidence established in DCMA 
Audit Checklist used 

Results of Quality system audit: 

e Notification to contractor 

e Notification to our customer 

If Quality System is found to be in compliance, DCMA will issue a 

“Statement of Qualification” to contractor 


DCMA STATEMENT OF QUALIFICATION: 


e Quality System Qualification 
e Name of Company Qualified 
e Applicable Standard 


e Document dated and signed by 
e DCMA CMO Commander 
e Copy of document sent to customer(s) 
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APPENDIX F. ISO PROCESS MANAGEMENT APPROACH 


The quality management system described in the revised standard is based on 
quality management principles that include the "process approach" and "customer focus.” 
ISO 9001:2000 and ISO 9004:2000, both, apply a process approach. The "20 element" 
structure of ISO 9001:1994 has been replaced by this process-based quality management 


system (PQM), which is shown schematically in Figure 3, below. 


CONTINUAL IMPROVEMENT OF 
QUALITY MANAGEMENT “—" Customer 


aso 9001) and 







Customer |,----__<-. 
aso 9001) and 


Parties 
(150 9004) 








Product 
= realizanion ZT} 


Figure 3. Model of a Process Based Quality Management System, [From Ref.48]. 


The adoption of these principles should provide customers with a higher level of 
confidence that products will meet their needs and increases their satisfaction (28,48). 
ISO 9000:2000 clause 3.4.1 defines a "Process" as any activity or operation, which 
receives inputs and converts them to outputs (26). The output of one process may directly 
form the input to the next process and the final product is often the result of a network or 
system of processes. Almost all activities and operations involved in making a product or 
providing a service are processes. Inputs and outputs may be tangible or intangible. 
Examples of inputs and outputs may include equipment, materials, components, energy, 


information and financial resources, among dhers (25). For organizations to function, 
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they have to define and manage numerous inter-linked processes. The systematic 
identification and management of the various processes employed within an organization, 
and particularly the interactions between such processes, may be referred to as the 
‘process approach’ to management. The revised quality management system standards 
promote the adoption of a process approach when developing, implementing and 
improving a QMS. ISO Standards are based on just such a process approach, in line with 
the guiding quality management (25). ISO allows the nature of the organization’s specific 
demands to determine how to apply the standards, but to perform activities within the 
process appropriate resources have to be allocated. A measurement system can be used to 
gather information and data to analyze process performance and input and output 


characteristics (52). 


PROCEDURE* 


(“Specified way to carry out an activity or a 
process” - may be documented or not) 





___input | = PROCESS =| output popucT 


(Set of interrelated 
Resources) or interacting activities") (Result of a 


(Includes 





process") 





YY 


MONITORING AND 
MEASUREMENT OPPORTUNITIES 
(Before, during and after the process) 


* Note -This is the definition of "procedure" givenin ISO 9000:2000. 
This does nat necessarily mean one of the 6 
“documented procedures" required by ISO 9001:2000 


Figure 4. | Schematic Representation of a Process, [From Ref. 48]. 


ISO 9001:2000 stresses the importance for an organization to identify, implement, 
manage and continually improve the effectiveness of the processes that are necessary for 
the quality management system, and to manage the interactions of these processes in 


order to achieve the organization's objectives. ISO 9004:2000 focuses on performance 
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improvements and guides the organization beyond the requirements of ISO 9001:2000. 
ISO 9004:2000 recommends an evaluation of the efficiency, as well as the effectiveness 


of the processes. 


The Plan Do-Check-Act (PDCA) cycle can be used to manage those processes. 
The "Plan-Do-Check-Act" cycle was first developed in the 1920's by Walter Shewhart, 
and was popularized later by W. Edwards Deming. For that reason it is often referred to 
as "The Deming Cycle.” Extensive literature exists about the PDCA cycle thesis readers 
are encouraged to consult other literature for a deeper understanding of the concept 


beyond the following paragraphs the writer includes (28,31). 


"Plan" establish the objectives and processes necessary to deliver 
results in accordance with customer requirements and the 
organization's policies; 


"Do" implement the processes; 

"Check" — monitor and measure processes and product against policies, 
objectives and requirements for the product and report the 
results; 

"Act" take actions to continually improve process performance;" 

Table 6. Table 6. P-D-C-A Defined, [From Ref. 25]. 


Within the context of a quality management system, the PDCA is a dynamic cycle 
that can be deployed within each of the organization's processes, and to the system of 
processes as a whole. It is intimately associated with the planning, implementation, 
control and continual improvement of both product realization and other quality 
management system processes. Maintaining and continually improving the process 
capability can be achieved by applying the PDCA concept at all levels within the 
organization. This applies equally to high-level strategic processes, such as quality 
management system planning, or management review, and to simple operational 
activities carried out as a part of product realization processes. The Note in Clause 0.2 of 
ISO 9001:2000 explains that the PDCA cycle applies to the QMS processes as follows 
(26,27): 


e Planestablish the objectives and processes necessary to deliver results in 
accordance with customer requirements and the organization's policies 
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e Do-implement the processes 


e Check-monitor and measure processes and product against policies, 
objectives and requirements for the product and report the results 


e Act-take actions to continually improve process performance 





FPDCA approacihaveto 
apply toe al desaibel 
processes in fig. x 


Figure 5. © The "PDCA" Approach, [From Ref. 28]. 


A second important quality management principle that is intimately linked with 
the Process Approach is the System Approach to Management, which states that 
"Identifying, understanding and managing interrelated processes as a system contributes 
to the organization's effectiveness and efficiency in achieving its objectives." Within this 
context, the quality management system comprises a number of interrelated processes. 
The processes needed for the quality management system include not only the product 
realization processes (those that directly contribute to making the product or delivering 
the service), but also numerous management, monitoring and measurement processes, 
such as resource management, communication, internal auditing, management review, 


and other processes (15). This can be seen schematically in Figure 3, which provides 
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greater detail of the kind of processes that typically comprise the quality management 


system, divided among clauses 4 - 8 of ISO 9001:2000 and ISO 9004:2000. 





Figure 6. | Schematic Representation of Typical Quality Management System 
Processes, related to Figure 3. [From Ref. 28]. 


Individual processes rarely occur in isolation. Outputs from one process typically 


form part of the inputs into subsequent processes, as shown in Figure 4 (25). 


Outputs from Outputs from Outputs from 
Process A Process B Process C 
—— | PROCESS A | ————_» | PROCESSB | ———~—_+ | PROCESS C | ——__» 
Inputs to Inputs to Inputs to 
Process B Process C 


Process A 
Figure 7. Chain of Interrelated Processes, [From Ref. 28]. 


"Continual improvement" requires an organization to focus on continually 
increasing the effectiveness and/or efficiency of its processes, to fulfill its policies and 
objectives. Continual improvement (where "continual" highlights that an improvement 
process requires progressive consolidation steps) responds to the growing needs and 
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expectations of customers and ensures a dynamic evolution of the quality management 
system (25,28). While the “continual improvement” concept of revised ISO 9001 is 
intended to promote the effectiveness of the quality management system, the revised ISO 
9004 is intended to improve the efficiency of the organization. Together they will help 
increase organization’s competitive advantage in the market, and enable them to better 


respond to its customer’s requirements (28). 


The consistency of the two standards helps organizations that wish to go beyond 
ISO 9001 to follow the guidance given in ISO 9004. An enhanced requirement for 
“continual improvement” has been introduced into ISO 9001, defining a complete cycle 
to improve the effectiveness of the quality management system (48). Continual 
improvement is a process of increasing the effectiveness of your organization to fulfill 
your quality policy and your quality objectives. ISO 9001:2000 requires that you plan and 
manage the processes necessary for the continual improvement of your quality 
management system (52). Continual improvement is not an easy concept to measure. 
Specifically in cases where product maturity has reached a high level or in many service 


oriented purchases. 


A new item that has been introduced into ISO 9001:2000 is the requirement for 
the organization to monitor information on customer satisfaction as a measure of system 
performance (48)."Customer satisfaction" is recognized as one of the driving criteria for 
any organization. In order to evaluate if the product meets customer needs and 
expectations, it is necessary to monitor the extent of customer satisfaction. Improvements 


can be made by taking action to address any identified issues and concerns (28,48). 
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APPENDIX G. ISO 9000:2000 RE-STRUCTURE 


A. BENEFITS AND IMPROVEMENTS 


ISO states a number of major benefits are associated with the revised quality 


management systems standards. Among them are (31): 


Applicability to all product categories, in all sectors and to all sizes of 
organizations 


Simple to use, clear in language, readily translatable, and easily 
understandable 


Significant reduction in the amount of required documentation 
Connection of quality management systems to organizational processes 
Provision of a natural move towards improved organizational performance 


Greater orientation toward continual improvement and _ customer 
satisfaction 


Compatibility with other management systems such as ISO 14000 


Provision of a consistent basis to address the needs and interests of 
organizations in specific sectors (e.g., medical devices, 
telecommunications, automotive, etc) 


The concept of the consistent pair - ISO 9001 covering the requirements 
and ISO 9004 for going beyond the requirements in order to further 
improve the performance of the organization 


Consideration of the needs of and benefits to all interested parties 


Main changes in the revised ISO 9000 standards are the increased focus on top 


management commitment, emphasis on the process approach within the organization, 


continual improvement together with enhancing satisfaction for customers and other 


interested parties (48). The main changes and new requirements that have been 
introduced in the consistent pair (ISO 9001:2000 and ISO 9004:2000) of quality 


management system standards are: 


The main changes (25): 


A new process-oriented structure and a more logical sequence of the 
contents 


A continual improvement process as an important step to enhance the 
quality management system 
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Increased emphasis on the role of top management, which includes its 
commitment to the development and improvement of the quality 
management system, consideration of legal and regulatory requirements, 
and establishment of measurable objectives at relevant functions and 
levels 


The concept of "Application" of the standard has been introduced (in 
clause 1.2) as a way to cope with the wide spectrum of organizations and 
activities 


A requirement for the organization to monitor information on customer 
satisfaction as a measure of system performance 


Significant reduction in the amount of required documentation 
Terminology changes/improvements for easier interpretation 


Increased compatibility with the environmental management system 
standard ISO 14001 


Specific reference to quality management principles. 
Consideration of the benefits and needs of all interested parties 


Addition of the concept of organizational self-assessment as a driver for 
improvement (ISO 9004:2000) 


The main new requirements include (25): 


Continual improvement 

Increased emphasis on the role of top management 

Consideration of statutory and regulatory requirements 

Establishment of measurable objectives at relevant functions and levels 


Monitoring of information on customer satisfaction as a measure of 
system performance 


Increased attention to resource availability. 
Determination of training effectiveness 
Measurements extended to system, processes, and product 


Analysis of collected data on the performance of the quality management 
system 


The primary aim of the "consistent pair" is to relate modern quality management 


to the actual processes and activities of an organization, including the promotion of 


continual improvement and enhancement of customer satisfaction (48). 
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B. ISO 9000:2000 CLAUSES 


ISO 9001 is organized under eight sections. The sections are organized into eight 


standard clauses. They are: 


Section 1 - Scope 

Section 2 - Normative Reference 

Section 3 - Terms and Definitions 
Section 4 - Quality Management System 
Section 5 - Management Responsibility 
Section 6 - Resource Management 
Section 7 - Product Realization 


Section 8 - Measurement, Analysis, and ImprovementOf the eight 


clause of the new standard, Sections four through eight are the most important to the 


QMS. These clauses actually comprise the “heart” of quality management system. Each 


one of these five main clauses has a number of sub clauses which support the concept 


identified in the main clause (32). 





91 





Figure 8. | Clause to Element Cross Reference, [From Ref. 33]. 

Elements, clauses, requirements are similar words used in much of the literature 
produced on the new standard, but they basically mean the same thing, areas to be 
addressed. All of the element requirements found in the “1994” standard are reflected in 
the “2000.” This does not mean that you will find a “word for word” transfer of the 
requirement to the new standard. What this means is that the “concept” that was covered 
has been retained, but not necessarily in the same form that was reflected in the 1994 


standard document. Please refer to Appendix B (ISO to MIL-Q Cross Reference). 


C. DOCUMENTATION 

Whole or entire re-structuring of an organization’s quality system or re-writing all 
procedures is not expected; however, the revised standards include some new 
requirements that will need addressing in a quality system. The three year 'transition' 
period allows accredited certification to the 1994 standards and ISO 9001:2000 to 
continue to coexist. This 'transition period’ will end on 15 December 2003. By that date, 
all organizations wishing to retain accredited certification will have to have migrated 


their quality management system to being compliant with ISO 9001:2000 (25). 


Basically two types of documentation exist. Those that tell us what to do and 
those that tell others what you have done (31). If an organizations current quality 
management system is successfully implemented, satisfies the needs and objectives of the 


organization, reflects the way the organization works, addresses all of the new 
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requirements, no changes are required. However, if the current documented system does 


not address all of the new requirements, additional documentation may be necessary. The 


International Standard ISO 9001:2000 has clarified the need for required documentation. 
Only 5 documented procedures, ISO 9001:2000 Clause 4.2.1, are required by the 


standards for administration of the system; however, other documented procedures may 


be required by your organization in order to manage the processes, which are necessary 


for the effective operation of the quality management system. This will clearly vary 


depending on the size of the organization, the kind of activities in which it is involved 


and their complexity (25). The quality management system documentation includes 


(31,48). 


Documented statements of a quality policy and quality objectives 
Quality manual 
Documented procedures required by this International Standard 


Documents required by the organization to ensure the effective planning, 
operation and control of its processes 


Records required by this International Standard 


Documentation 







Pyramid Philosophies 
Quality Manual and 
Policies 
Procedures Principles 
and Strategies 
Work 


Instructions 


Records 


Figure 9. | Documentation Pyramid, [From Ref. 33]. 
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APPENDIX H. INTERVIEWS 


A. INTRODUCTION 

The information presented in this study was gathered through a sampling of 15 
managers, directors, or auditors of quality assurance management departments from 
companies that are currently ISO 9000:1994 registered and the DCMA. The private 
organizations were selected based on their prominent role in the defense industrial base. 
DCMA representation was selected based on their lead role in organizing and distributing 
guidance and training on ISO 9000 and expert knowledge of military quality assurance 
and management. The aggregate list of respondents is presented in Appendix A. 


B. INDUSTRY INTERVIEW RESPONSES 
1. Question One (ISO 9000:2000 Registration) 
Are you currently a registered ISO 9000 organization, and if so, for how long? 
a. Discussion 


The purpose of this question was to understand the background of the interviewee 


and the organization as it pertains to ISO Registration. 
b. Response 


All of the interviewees were currently or had in the past been part of ISO 
9000:1994 registration and with DCMA Audits. One interviewee’s current department is 
not ISO registered but maintains a MIL-Q 9858 QMS. 


Z. Question 2 (ISO 9000:2000 Registration) 


Is the selection of sub-contractors biased to those registered ISO 9000 compliant 


and will you require those suppliers to become ISO 9000:2000 registered? 
a. Discussion 


This question was asked to query industry on its supplier or sub-contractor base in 


an attempt to explore industry ability to qualify suppliers or vendors QMS. 
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Reponses 


No, our customers require that we have a small Disadvantaged businesses 
program (based on a percentage of contract dollars) for procurement of 
supplies and services. Many of these suppliers are family owned and run 
businesses. The imposition of the ISO registration process on these 
suppliers could cause financial/technical barriers on them. To alleviate the 
problem we perform audits to insure that the supplier has sufficient control 
in-place to insure that product provided to GDLS meets or exceed our 
customer’s requirements. 


90% of our suppliers are ISO 9000 registered. In order for our suppliers to 
maintain their registered status with their registrar they must be ISO 
9001:2000 by the end of 2003. The internal planning and change process 
to evolve to the new standard is very comprehensive and driving the 
requirement before the mandatory transition process is required would 
drive cost into programs that are being squeezed on a yearly basis for cost 
reductions by our customers. Not one of our customers is willing to pay 
more today for improvements even with the promise of future gains. With 
the current 2 to 3 year cycle of Program Managers assignments this is 
unlikely to change in the future. 


Registered suppliers avoid some of the obstacles required to become part 
of approved supplier base, however, registration is not a pre-requisite. 


No, it is not biased in that direction. Our processes and procedures allow 
for acceptance of a supplier based on the systems and processes in place 
that will allow for control of the material manufactured for our customers 
and us. In the meantime, the current plan is to move those suppliers 
currently on our Approved Supplier List (ASL), as well as those who wish 
to do business with us, towards process control and system administration 
very similar to that found in the QS 9000 standard. 


We perform a supplier assessment based on ISO 9000 but do not require 
formal (3"! party) registration. Although our registration is now to the 
2000 version, our supplier assessment will continue being performed to 
the 1994 revision until approximately 2003. 


Yes and no, if they are ISO registered we may not conduct a self-survey or 
not as stringent of a survey, that is the advantage to the supplier and us. 
We still deal with many MIL-Q and MIL-I companies that have not 
converted to ISO so we may still need to conduct other survey methods. 
Unless the purchase order requires it we will not. 


We have established an advance quality system with our suppliers based 
on ISO 9000 and then they are told to comply with the system. We do not 
require they receive 3d Party registration. We will conduct a 2d party 
review. Past performance plays the critical role in our supplier base. 
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e No we do not require ISO we do encourage it. We evaluate their quality 
system approach. We look at the quality manual and quality system, past 
performance is a big part of it. We do not stipulate ISO. The advantage to 
an ISO registered organization is we may and often due reduce or 
eliminate our periodic inspection of a supplier. 


e We have no bias towards organizations that are not ISO. If they are we 
may waive some of our inspections and surveillance but much of that 
depends on past performance and the status of the supplier. 


e Our company uses a Quality Approach Supplier Relationship. It is 
rigorous quality assurance approach. A Quality System Review is used to 
identify potential suppliers to ensure the potential supplier has in place 
appropriate controls to meet Motorola standards. We do not mandate 
suppliers use ISO or become ISO certified since our own quality assurance 
provides greater detail and insight into a supplier. ISO does provide some 
benefits but the quality evaluation is based more on our relationship with 
the supplier, past performance, length, and fulfillment of previous 
contracts. 


3. Question 3 (ISO 9000:2000 Registration) 


What planning or guidance is your organization conducting to meet the ISO 
9000:2000 certification and what are the significant changes it imposes upon your 


organization’s quality assurance/management program? 
a. Discussion 


This question was designed to understand the planning and _ guidance 
organizations are using to meet the new standard requirements. The question effort was 
to gain a better understanding of any common themes or issues reoccurring between 


various industries as they plan and execute ISO 9000:2000 implementation. 


b. Responses 


e We are currently reviewing the standard and will implement according to 
the guidelines and timelines laid out by our registrar. There are no 
identified significant changes that we will need to address as a result of 
registration to the new standards, although we will be taking advantage of 
the opportunity to more fully map our processes and change our work 
instruction and procedure formats. If we find that there is more significant 
impact to our company we will find the necessary support to secure 
standard realization... 


e We established a 9K2K (ISO 9001:2000) transition team in December 
2000, with representative members from each facility throughout the 
company. ‘This team was given various forms of formal 9K2K training 


97 


and subsequently developed our corporate ISO 9K2K transition plan. The 
teams major transition compliance activities can be summarized in 3 
categories: 


0 Re-write the Quality Manual. 


re) Perform documentation gap analysis and revise 
procedures/instructions as needed. 


0 Develop & Rollout employee awareness and delta training. 


Because of (what we believe to be) a very robust implementation to the 
previous 1994 standard, our gap assessments indicated that our 
management system was already in compliance with the 2000 
requirements. However, making some modifications to existing 
Instructions to facilitate the availability of Objective Verifiable Evidence 
for some of the new requirements (e.g., Documentation of measurable 
improvements) needed to be made. Additionally, significant effort needed 
to be expended in mapping our existing documentation to the requirements 
of the 2000 standard. 


Yes we are moving towards 2K (ISO 9001:2000) registration, we are 
neither encourage or discouraged from DoD to make the conversion or to 
push our suppliers to convert. There is no significant or radical changes or 
costs we need to make with our current system to meet the 2K 
requirements. I do see it as a big change when you compare the 94 
versions to the 2000 version. For me to explain how I see the change let 
me provide a brief history of ISO as I see it. The 87 ISO emerged from 
the European Common Market and used ANSI, MIL-Q, British Standards, 
and ASMI Code. So there was nothing new about ISO. What was unique 
about ISO was the registration process. It was used as a barrier for entry 
into the common market. The registration framework makes it successful 
and perpetuates the success, financially, of the companies that are 
registrars. It is a product or service the registrars provide for sale. ISO is 
a commercial product, not a significant customer support. 


Our transition is at two levels at the enterprise and the business unit level. 
The company is developing an overall enterprise 2k-transition plan, but 
each business unit has their own implementation plan. We have many 
units at 2k compliant already. We are taking the lessons learned and 
corrective actions they have made to conduct our transition to 2k. Our 
focus in preparing for the transition is on our business units processes 
looks at the corrective actions taken at other business units and to conduct 
on-going improvements to our processes. 


Our transition plan is to meet the requirements by April 2003. We will 
seek a joint registration between two plants. We will focus our plan on 
what we see are the significant changes in customer satisfaction 
measurement, document our continuous improvement process, and update 
our quality manuals. We are looking at the measurement and analysis of 
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4. 


our critical processes and goals connecting them to objectives and targets 
for improvement and then continually monitoring that process to continue 
to improve. 


I see the ISO changes as significant. A heavy emphasis on continuous 
improvement, management oversight, record meeting. It places more a 
burden on contractors to document their continuous process improvement. 
I disagree that the five disciplines talked to in the new ISO were already 
built into the previous 20 elements. The IOS provide little guidance on 
how to evaluate the new requirements. What is the right process to 
evaluate continuous improvement? Looking from the quality systems 
perspective and implementing 6 Sigma, training, and resources in all the 
departments across an organization. ISO focuses on how to implement 
and integrate all those processes across you organization. The 20 elements 
are those things that focus on product quality and you cannot do with out. 
The new version includes those but addresses them towards a global 
implementation. 


Question 4 (ISO 9000:2000 Registration) 


Do you believe the costs (financial and time invested) associated with becoming 


ISO 9000:2000 registered beneficial (more profitable and better quality product) for your 


company and your customer (DoD and other)? 


a. 


Discussion 


The cost of registration and its benefits are an obvious start point to discuss the 


benefits, if any, ISO 9000. The researchers interest was to focus the value added to the 


organization, as it would compare to an organization implementing its own unique QMS 


or the MIL-Q standards. 


b. 


Responses 


I believe that the implementation of ISO led us down the current path that 
is in place and that has resulted in our company being more competitive 
and profitable than had we not achieved ISO certification. That said, I 
believe that the Management at my company is committed to continuous 
improvement and providing the highest quality product in the marketplace 
while fulfilling and exceeding the expectations of our shareholders. This 
would be true regardless of the Quality system we have in place. 


Having previously been a Mil-Q-9858 company, the cost trade off 
between ISO registration activities (including internal audits and other 
internal efforts) was equally counterbalanced with the absence of frequent 
Government/customer quality system evaluations. 
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Unfortunately there seems to be a growing desire on the part of our (nom 
Government) customers to perform their own Quality Management 
System audits, irrespective of the fact that an accredited 3rd party registers 
us. These audits are causing us to evaluate the benefit/savings of being 
registered. 


The largest advantage that we experienced with the ISO 9000 Standard 
(which ultimately translate to being more efficient thus more profitable) is 
the flexibility that we (the supplier) are afforded in making our own 
determination as to HOW the requirements will be accomplished. The 
legacy Mil-Standard was quite prescriptive in not just WHAT but also 
HOW requirements would be satisfied, removing most of the companies 
ability to be agile. 


ISO is not value added to our organization. It is more of a marketing tool 
than an improvement to our quality system. Our registrar’s advice and 
surveillance have been of little use. Our business has in use a 
sophisticated and complex quality system and it would be very difficult 
for any registrar to provide a great deal of value towards our system. 


The most beneficial outcome of ISO is to have an outside registrar to 
validate the quality cultural changes that need to be made in our 
organization. It does not require the quality department, often thought as 
the quality police, to champion the change. The change required is 
organization wide. It is difficult for an organization to make quality 
changes internally. ISO helps us focus on our programs we want to 
establish a refocus on what we are doing (competencies). The 1994 ISO 
helped in our process focus moving from the previous MIL-Q. ISO 
provides a competitive advantage bit only short-term. Many organizations 
have known for decades what they needed to do but were unable internally 
to do so because of the culture. ISO provided the focus or catalyst for 
change. 


Yes and no, before ISO we had MIL-Q. There are some differences but 
there are many common requirements. ISO brought in some other 
standards like human resource management, contract review and internal 
audit process and how to deploy those resources properly in the 
organization. ISO expands your business and defines your activities that 
support ISO. The biggest benefit is the cultural change it can bring about. 
What we once thought as unrelated to quality is actually related. It effects 
the whole organization. Previously we looked at the downstream process 
ISO brings in the upstream processes that impact the product. Quality 
goes to management commitment, capable processes on the manufacturing 
floor. ISO can bring in the discipline to change. I see no significant costs 
to change to ISO 2000. They are asking for things already implied in the 
1994 standard. 


A few years ago we conducted a cost analysis and breakout of ISO 
registration. The largest barrier was in training. The time it took to train 
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3: 


the workforce and the time it took away from their normal duties. ISO 
registration and preparing for it is very human resource intensive. 


The ISO quality assurance process to follow ensures some consistency, 
and offers an appropriate methodology to use. The ISO registration 
provides leverage as a sales marketing tool and to some a competitive 
advantage although ISO is becoming more prevalent across industry. The 
third-party registration of an unknown supplier does provide some benefit 
to the buyer that some baseline of a quality system exists in that potential 
suppliers organization. Some small businesses have difficulty in justifying 
the cost to get a third-party registration. Third party registration provides 
another benefit as well. Organizations often self assess with ‘blinders’ on; 
they think they are doing well but may not be. The third-party registration 
provides a baseline or benchmark to compare you to other organization 
and a standard or template that is industry accepted and proven to work by 
an “unbiased” independent party. The cost of our last registration, just the 
visit, was $25,000. This cost does not measure numerous other resources 
used for the registration. 


Question 5 (ISO 9000:2000 Registration) 


What are the major barriers in becoming ISO 9000 certified? Are there new 


barriers introduced by the 2000 version? 


a. 


Discussion 


This question’s aim was to outline what impacts or obstacles implementing ISO 


9000:2000 has contractually and against resources. 


b. 


Responses 


The major barrier to become certified is having the required time for 
everyone to provide his or her input. Everyone in the organization has his 
or her normal job functions and finding the extra time can be difficult. I 
don’t see any barriers in the new revision. 


For a defense contractor the 2000 version did not impose anything that we 
were not already doing. The added requirements required us to map those 
activities to our manual. 


The major barriers encountered at my business unit were associated with a 
new way of looking at Quality as a team, the “paradigm shift,” if you will. 
The increased emphasis on management involvement for the latest 
standard will pose no significant difficulties. 


The initial barriers (I prefer the term challenges) to ISO 9000 certification 
are: 
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6. 


An initial and significant effort in mapping or restructuring one’s 
documentation system to align with the standard. 


An initial increase in the need for employee awareness training due 
to additional requirements (e.g., Familiarity with the Quality 
Policy, Knowing who the ISO management Rep is...) not 
previously thought to be necessary. 


Establishing the budget for training internal auditors and hiring a 
registrar. 


Added challenges related to the 2000 Standard: 


oO 


Designing a system to capture evidence of improvement programs 
and upper management’s involvement. 


Re-calibration between us and our registrar regarding some of the 
new requirements and exactly what they mean. (Based on the 1987 
to 1994 transition, the variation of interpretation reduces with time; 
but the initial differences in opinion starts out fairly wide) 


The culture of us versus them, between the quality inspectors and 
production, is difficult to change. The culture change needed is the 
move to make everyone responsible for quality assurance in the 
organization resulting in a better product. Making each one focus 
on their processes in quality management so we don’t have the 
performance and defect problems to begin with. The goal is to 
make sure all the various partners in our company take 
responsibility and accept their function for the quality of what they 
do whether it is a process or a product. This helps every one 
understand that quality is everyone’s business. 


The major barrier is cost. Our organization has no management barrier. 
Our leader are versed in the quality control process exceeding ISO in 
many ways, like our 6 Sigma Manufacturing and Services. 


Question 6. (Quality System and Quality Product) 


One criticism of ISO 9000 registration is it provides no guarantee of product 


quality or conformity, only that a quality process is in place. Do you agree, disagree with 


this statement? 


a. 


Discussion 


This series of questions is based on the recent incidents of ISO 9000 registered 


organizations producing non-conforming and defective material. Its purpose was to elicit 


comments on some of the problems or misconceptions about ISO registration and what 


measures can be taken to increase quality control and product conformance. 
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Responses 


No registration process will guarantee product quality. The guarantee 
audit would be so expensive that it would be unaffordable, in addition, 
there are very few people that could perform an audit to the scope and 
detail required to verify every step in every process is acceptable. Even 
100 percent inspection is only 85% effective. 


This is correct, however the nonconforming procedures and the corrective 
action procedures should be finding the nonconformance and corrects the 
deficiency. 


I agree. A great process must have discipline in place to achieve the 
desired results. 


This would be true of any quality standard. It is not nearly as difficult to 
achieve certification, as it is to implement the underlying concepts behind 
the process. 


I agree. ISO registration indicates that you have the minimum necessary 
business/management processes in place; it is the starting point for stellar 
quality not the zenith. 


If you really understand that quality is everyone’s responsibility. If you 
understand the essence of the processes and that is what Process Based 
Management helps you to do. The fact that everyone is looking at their 
processes through established metrics, and measuring the process against 
the metrics, you would be hard pressed to have a major systems failure. I 
see the problem not with the registration process. A company may not 
have bought into the quality process they were registered against. I am 
speaking again of culture. You can put the quality system in place, get 
registered, but if you are not conducting the process management or if 
everyone is not taking responsibility for the established process you will 
have problems. 


It comes down to the management philosophy of why you are doing this. 
You can fool the auditors but if you use them as a tool to improve your 
process you can use that information to make real improvements with your 
company. People of the organization must embrace the need for their role 
in the quality process of the whole organization and its product. Key is to 
understand why we are changing the culture. It is a company generational 
change. What I like about ISO is how it grows with the evolution of 
quality standards and shifting culture over time. We take it in phases. It 
helps us evolve, our culture, management, and people. 


I agree, no matter what system you use you could be making good scrap. 
The 2000 standard says you have to have measurable targets or objectives 
linked to continual improvement. An organization’s policy objectives, 
cumulative resources, and capable processes along with ISO result in 
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Cc. 


quality. There is no quality guarantee with mere compliance to ISO. ISO 
provides a disciplined approach to maintain you system. 


If you use the analogy of a car air bag to that of a quality system like ISO 
or MIL-Q, statistically you may never use a car air bag but the time you do 
you are glad you have it. A quality system provides that check and 
balance for the time when a quality issue or product default presents itself, 
thus reducing the times you create defective material. Quality addresses 
risk. Well-defined, well-implemented quality systems greatly increase the 
percent of time you have no defects. The quality system has parallels to 
airport security. No system can guarantee that no security breach will 
occur except one where no one would want to fly due to all of the 
constraints and restrictions. You want to have a system that provides the 
greatest return on quality to the cost of producing the product to that 
standard determined by the level of risk you are willing to assume. You 
want to get your highest return on the cost of the process, control 
measures, and organizational resources you place on a quality system. 


ISO 9000 is just an audit of your quality process providing a registration 
that an organization is meeting or following a quality system in place, not 
necessarily if the process is being properly implemented. To some degree 
ISO provides product quality assurance but there are other intangibles: 
watranty, consistency, quest for quality, process tools to continually 
improve, cost analysis, and product process quality issues and resolution. 
No organization is free from quality product issues. There are always 
concerns with suppliers, safety, etc. The critical part is the feedback loop 
and internal and external to our company to provide quality products. 


Question 6a 


Is it possible for any preaward survey of quality assurance to detect indicators of a 


sub-standard product or service? 


Yes, remember that the ISO 9001:2000 standard is a Quality Management 
System. You also need to know that there were many debates about 
taking the "Quality" out of the title during the development process. 


No, on a small insignificant matter. Yes, if the system has a major 
breakdown. 


No, a preaward survey will only indicate the presence of capability, and 
will not provide assurance of the actual product/service quality 


The preaward survey can provide a companies core competencies and 
capabilities. It can detect a critically sub-standard quality. Will it detect a 
potential bad product or service? I say it depends on the qualifications, 
skills, and insights of the auditors more than any survey or checklist. 
DCMA, at best looks at an organization in-house, for two days for a 
preaward survey, it can only provide a good basic feel. 
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d. Question 6b 


What guarantees does ISO 9000 certification provide towards a quality achieving 
organization resulting in a quality product? 


e The certification is not the key internal process management and controls. 
The management system including internal audits are key to an 
organization achieving a quality product. We have averaged about .5 
minor nonconformances across all of our facilities and all audits since 
1997. 


e Any organization can put together a system that can attain registration; the 
key is the desire to perform at half throttle or at full throttle. The greater 
the effort is, the greater the return. 


e None, I do not believe that ISO 9000 is a guarantees quality; rather it is the 
single best indicator of the ability to produce a quality product. 


e ISO procedures are not intuitive and are not directly related to the product. 
ISO procedures do not yield a product; good procedures or good 
requirements do not necessarily yield a good product. It is more directly 
related to the process. We have systems audits, business controls, ISO, 
manufacturing techniques, SPC, etc. The degree of compliance is 
measured. Do we have perfect compliance to our process procedures, and 
if we do have perfect compliance then do we have a good product. For 
example, do we have a good operation but the patent dies. The true test of 
process procedures is not if they meet some artificial requirement like 
ISO. The test is, “Does the product work!” If you have processes and 
procedures that produce good product then you have something. You then 
establish those procedures in writing and then train to them and measure 
them. The existing procedures become experience retention in your 
workforce. Experience retention and experience culture make a good 
organization. This experience is put back into your continuous process 
improvement and control. 


e. Question 6c 


What value added does the ISO Registration have? 


e A second set of eyes, a regularly scheduled review that forces you to put 
things in order twice a year. And the cost is a management commitment to 
support the management system. 


e Our registration helps to maintain our customers’ confidence and that of 
the public. 
e It sets a direction that is up to the team who implemented and achieved the 


certification to understand, and allows for the base lining of improvements 
achieved through continuous improvement efforts. 
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When recognized by the customer, ISO registration eliminates the need for 
customer assessments; yielding instead to certified 3" party audits. This is 
a saving to the customer and to the contractor in that a single registration 
audit can eliminate multiple customers performing similar duplicated 
assessments. 


An ISO 9000 registration is commonly understood across the industry. 

Where in the past a thesis was expected to articulate your quality system; 
now simply showing ISO registration is understood and accepted. I equate 
ISO 9000 registration to a college education. In the same sense that a 
college degree neither means that the graduate is ‘smart’ nor does it mean 
that the graduate can function as an experienced practitioner in his/her 
field; the education does demonstrate that the student has successfully 
demonstrated the ability to learn and possesses the basic understanding of 
elements of the field that he/she has studied. 


Similarly, ISO 9000 does not mean that a company produces a great 
product; but has demonstrated that it possesses all of the basic processes 
deemed necessary in order to produce a quality product/service. 


The value added is the industry wide recognition of the standard. 


Question 7. (Quality System and Quality Product) 


What quality assurance/management measures do you take to insure your own 


process and that of your suppliers demonstrates product conformity and/or critical 


characteristics, in terms of defect levels, are acceptable; and that improvements are 


planned and later demonstrated? 


a. 


Discussion 


This question was asked to generate responses to address issues raised in the 


previous question on how to address quality assurance above and in addition to ISO. 


b. 


Responses 
We use trend analysis to show improvements. 


We have a broad range of measurements from sampling, SPC 100% 
inspection, etc. 


Internal and external audits are the most common means of system and 
process validation, although a sampling of material is also a must. We 
also monitor material at all phases of manufacture and note any 
deficiencies and associate those deficiencies with a responsible party 
through root cause analysis. Corrective actions are reviewed and accepted 
and validated in each case where failure has occurred. 
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Product evaluations are the major “added” activity to those already 
addressed by the ISO standard which ensure product conformity, 
critical/key characteristic compliance and that the deliverable product is a 
quality item. 


We use many tools, SPC, ISO as a baseline in some areas, varying trend 
analysis etc. The key to success is management’s ability to know when to 
use the right tool. 


We have advanced quality procedures levied against our suppliers. They 
focus in three areas, variability control, hardware quality audits, and ISO 
process management approach. Internally we have a constant source of 
metrics that we manage with DCMA, internal management, suppliers and 
each business function. We have transitioned the metrics from a product 
focus where we tracked results and defects to more up front in the process 
measuring something that will not allow us to have the defect. 


We use our supplier evaluation selection process. It measures and 
monitors suppliers processes and controls, involves a supplier rating 
system and provides procurement quality assurance. It monitors our 
suppliers through onsite inspection and upon receiving a product. The 
evaluation or rate dictates the amount of surveillance and oversight we 
use. 


Other measures have been in developing a supplier corrective action 
system where we have partnered with our suppliers to develop a better 
quality product. We have reduced our supplier base to gain better control 
and we hold an annual supplier symposium over three days to discuss 
future business plans and operations. 


In-house we use noncompliance control corrective action. We have 
quality improve ment boards for nearly every business function and unit. 
These boards use the experience and information from the people actually 
implementing the process to make improvements. Information and 
improvements from our quality improvement boards that meet weekly and 
our quality improvement council that meets quarterly involve all 
stakeholders to improve our organizations quality improvement process. 


The key is to have a flexible and dynamic system, whether it is ISO or 
MIL-Q, that responds to the conditions at hand. The system needs a list of 
menu options or courses of action to follow to implement to distinguish 
between a trend and an isolated incident with a vendor. 


We have many pre-ship quality processes. Most are based on the CMM 
Model and follow CMI and our Standard Processes. We have a process 
that looks at and measure the critical paths of a product; design, build, 
manufacturing, test and function, focusing on the products “critical 
characteristics.” We also use break and fix, accelerated life testing to 
mature our products to measure quality level. 
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On the production and service side we have numerous management 
review, various avenues for customer feedback, tracking systems like 
systems defect trends. All of these procedures are linked to our 
organization’s Failure Review Board involving all product players 
(stakeholders). Suppliers and customers are a part of the entire process 
and manufacturing relationship. An output is our Quality System Report 
Card providing our process to improve. 


Question 8. (Quality System and Quality Product) 


Have the problems, shortcomings, and lessons learned with the previous ISO 
9000 standards and registration been addressed under ISO 9000:2000 and have the latest 


technologies and best business practices been introduced into the new ISO standard? If 


not, what are your recommendations on what should be addressed and introduced? 


a. 


Discussion 


This question was used to allow organizations to provide the researcher with 


insight into what quality assurance practitioners saw as the strengths, weaknesses, and 


opportunities of both ISO 9000:1994 and 2000 provide. 


b. 


Responses 


They did a great job. Could there be improvements? Always! Getting 
consensus from 100 countries and thousands of representatives is near 
impossible. Let the evolution to this standard happen, I think we will find 
that the Registrars and Companies will incorporate the standard in the 
spirit of it's intent 


The document (ISO 9000:2000) is biased towards the supplier and what it 
takes to meet the document. Suppliers have the most influence on the 
document. It is tailored for them. What are the buyers risk when they 
purchase something? The buyer community has not been addressed in the 
2000 version. The customers of the registrars influenced ISO 2000. I see 
it as a less effective document and aimed more towards the service 
industry rather than manufacturing. 


No, our requirements always have and always will be more stringent than 
commercial industry, not to mention we have our customer living within 
our facility. 


In my opinion, the new standard has complicated process. The new 
standard will create a reduction in registration of small companies (500 or 
less) due to the additional requirements, which will increase business cost. 


I do not believe a standard should dictate technologies and business 
practices but rather should provide a framework for Management to 
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determine what works best for the particular product manufactured and 
sold. The previous [SO standard built the foundation, and it was ample. 
The latest revisions appear to require added management involvement and 
recognize the cross-functional requirements that are necessary in an 
optimally running company. What has occurred in my eyes is the standard 
has grown to better reflect the current business environment. 


The only improvement that I see in the 2000 version is that it is 
restructured to eliminate the perception that ISO 9000 is 20 separate, stand 
alone elements that “the quality guy” is responsible to ensure are met. 
Unfortunately, a large degree of vagueness in the requirements of the 
standard accompanies the new revision. 


ISO 2000 is a significant and radical departure and move towards supplier 
ease of implementation as opposed to effectiveness as a customer tool. 

The new version is more concerned with producer risk than consumer risk. 
The ISO standard is neither good or bad, it is not a silver bullet of quality 
to an organization either. I would not give the ISO document to a new 
company to begin business. It is not a specification but more a course in 
general management. I see three new areas in ISO 2k, continuous process 
improvement, quality objectives, and customer satisfaction. While these 
are new to the document does it help be make better widgets? The whole 
focus of the three is nothing new any good business is doing these things 
already or they would not be or continue to be in business. The document 
forces customers to provide feedback. Why should ISO force feedback? 
Every time a customer returns to me they are providing feedback. An 
organization cannot remain in business if they do not receive feedback. 

The document should be intuitive for any competitive market in a 
capitalistic society. ISO provides no edge it is generic it wants everyone 
to be gray. It lends itself better to socialistic markets where competitive 
prices of the market place are not allowed to establish the market. Our 
markets (U.S) require organizations to have an edge or advantage to 
prosper. ISO provides only a baseline and should not be used to 
distinguish between what an organization is to use and how it will provide 
the best product or service. Do we want a level playing field of quality 
management? Should every one be on an equal basis or in a standard 
way? Our ISO registrar criticized the way we managed some areas stating 
it was not like everyone else did it or as the ISO document prescribes. My 
reply is, “Exactly that is what provides us an edge over our competitors 
and provides us more of a market share and/or return on investment.” It is 
our quality management secret and strategic advantage; do not give it to 
other companies you registrar. I would not recommend that a company 
follow ISO and nothing else because it leaves them equal with every one 
else. Businesses succeed or fail on their ability to manage. There is no 
quality; quality management is an illusion invented by academics and 
registrars who want to create a discipline where one does not exist it is an 
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art. An organization and its people must be competent, capable, 
responsible, and accountable for doing the right thing. 


The major changes we see in the ISO 2k is its focus on process-based 
management. For our organization the program is to identify our critical 
processes, develop a plan, and use our sever-step process to evaluate the 
process and improve it through the monitoring process. Our sevenstep 
process of development fits well into the ISO standard and its process 
based management approach. Through understanding your process and 
developing metrics around that process the measure how you are 
performing is how to continue improvement in the process. Previously 
ISO was a system that looked at content and processes based on a process 
now ISO focuses on process quality ownership. 


There are explicit and implied changes in the new ISO most are adapted 
from the process business management approach that asks for an 
organization to define its core competencies, relationships between each 
process, what the process flow is, and the criteria if capable or not. I see it 
failed to address much on safety. It does look at the work environment but 
many define that very narrowly. 


I do not see many shortcomings. What is obvious to some in the previous 
20 elements is not so to others. What organizations see as shortcomings or 
weaknesses and even strengths of ISO is largely dependent on the culture 
of the organization implementing ISO. It is largely dependent on the level 
of importance and role culture plays in allowing the organization to 
understand ISO techniques of quality assurance and management 
resolution to champion the organizational change to the current culture. 


Question 9. (Quality System and Quality Product) 


Will ISO 9000:2000 emphases on customer focus, product realization and 


conformity, process management, and resource management address the past shortfalls 


associated with the 1994 ISO version? 


a. 


Discussion 


The question was designed as a follow up to the previous question to address the 


effectiveness of some of the new changes built into ISO 9000:2000. 


b. 


Responses 


Yes, it will help to cause continuous improvement in those and other 
areas. 


No, I believe that the attempt to add these emphases added a greater lack 
of clarity. All of these ‘elements’ existed in the previous version. 
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I have three main criticisms of the new ISO document. One is its focus on 
producer risk rather than customer risk. The first three section of the 
document (ISO 9001:2000) are confusing and unclear it is not until you 
get to section four that you find any substance. The dilution of the 20 
elements into the new five area of quality management is a Harvard 
Business School course on how to manage, not a good quality system. 
The 1994 version provides a checklist of the basic fundamentals of 
management; however, in the 2000 version all of that is gone. The 1994 
version provided a quick way, or a rough-cut or view, for customer 
auditors to evaluate suppliers. The new standard is deplorable because it 
is harder to determine what it is requiring. 


The previous 20 elements are now spread against the new five disciplines 
plays well into how key stakeholders in quality management development 
are reorganizing. It moves away from the stovepipe approach of 
organizations where one or some function was responsible for one or a 
series of the elements. ISO now spreads the previous 20 elements 
responsibility across the entire organization using the five disciplines. 
You cannot do away with the 20 elements; they are the basics, the 
premises of business. ISO did not invent a total new way of doing 
business; just new ways to better develop your own processes. 


My concern with the 2000 version is the customer satisfaction 
measurement as it applies to organizations that primarily deal with the 
DoD as a customer. Customer satisfaction is not an easy thing to measure 
when DoD is your customer. We have many customers, DCMA, DCAA, 
the buying command, the program office, the systems manager, and 
ultimately the soldier. ISO does not tell you how to do it just that some 
process needs to be in place to estimate it. 


Question 10 (DoD and Contractor Quality Assurance Relationship) 


Has ISO 9000 become the quality management standard for DoD contractors, 1.e., 


must a contractor become ISO 9000 certified to conduct business with the DoD 


especially in contracts requiring a higher-level quality standard? 


a. 


Discussion 


The issue the researcher wanted to raise in this question is, although ISO is not 


mandated by any Federal procurement regulation and guidance, has the ISO influence 


created an environment in which industry must be compliant with ISO to conduct 


business. 
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Responses 


Not that I am aware of. Nor should it be. From a competitive standpoint 
the DoD should compete its contracts, as any good business would do. It 
should insure that the material it requires meets its requirements and that 
the place of manufacture is consistent in its performance. Some 
companies are quite capable without certification to meet any customer’s 
standards and expectations. If all contractors were required to be ISO, 
would that eliminate those who are registered to other standards, QS 9000, 
for example? Being certified and maintaining that certification through 
semi-annual audits helps to mitigate risk to the customer, and the DoD is 
after all a customer. A company’s commitment to high quality is not 
simply reflected in its registration. It is reflected in its workforce and 
product. 


What concerns me is the use of ISO 9000:2000 in contracts and to qualify 
my organization as a supplier under that standard, especially the measure 
and aspect of continuous process improvement and product realization. 
Those are both difficult to measure contractually. For example, lets say 
you manufacture bicycles and you have made bicycles for years. You 
initially improved from 90% to 95% then 95% to 99%. Know you are at 
the 99.99% you will have little, if any, improvement. Does that mean I 
failed to meet my contractual obligations? Will I be terminated for not 
improving my product? Improvement is a good thing but not an easy 
contractual element to accurately measure. The key is to provide 
motivation for the supplier to improve on the contract not to punish the 
supplier if they do not. Motivated suppliers to improve and provide good 
products. Contract risk is mitigated through the motivating the supplier 
through contract. 


Some would say yes, the way contracts, statements of work are written it 
has become a de facto requirement. Of course it is always dependent on 
the contract and what an organization is using the contract to purchase. 


No, it can be ISO or equivalent. MIL-Q, QS9000 that is more restrictive 
and prescriptive can also be used. 


No, in fact in November of 1999 we participated in a program to update 
MIL-Q 9858 checklist and be evaluated against the revised checklist. The 
reason DCMA wanted to revise the past checklist was the realization that 
many DoD contractors wanted to remain and still use the MIL-Q standards 
and did want to move to ISO. Many organizations were and continue to 
work well under the MIL-Q standards and could not justify the cost to 
switch to a new system. MIL-Q is more directives than ISO is but ISO 
contains many “shalls” and “wills”. 
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Question 11. (DoD and Contractor Quality Assurance Relationship) 


Should DoD impose ISO 9000 standards and registration on contractors and sub- 


contractors as an entrance to conducting business with the DoD? 


a. 


Discussion 


This question was meant to elicit general comments on the benefits and problems 


of a DoD mandate of ISO 9000. 


b. 


12. 


Responses 
No, this puts a financial burden on small companies. 


All contractors have a system that meets or exceeds this due to the nature 
of our business after all ISO is an offspring of 9858. If the DOD were to 
impose registration then defense contractors would be able to bill the 
DOD for our ISO registration cost. This will not happen. 


Yes, but if 3" party registration is imposed, DOD audits should be 
removed. 


In my opinion, DoD cannot use the blanket policy of choosing only ISO 
registered organizations to do business with. Commercial buyers and 
producers must be better than that ISO). Selection must be process and 
skill determined and situation dependent. You want to use source 
inspection and performance data and feed that into a process of source 
selection. DCMA and DoD has not done a good job of doing that they 
have rot invented the proper tools to use to make that process work as 
well as industry has. A good example of how this works is a vendor- 
rating program. Product is inspected; performance data is generated and 
processed into our vendor-rating program across the organization. We use 
it as a tool for future supplier selection. I see DCMA request the data but I 
don’t see a documented transfer or process to provide the information 
across DoD organizations. 


In general terms no, but ISO has influences on all of these. 


Maybe, DoD should look at whether the standard meets what they are 
buying. Most contracts say ISO or equivalent, most of our international 
contracts specify ISO by name. With our international business we must 
have ISO. 


Question 12. (DoD and Contractor Quality Assurance Relationship) 


Some DoD Quality Assurance Representatives (QARs) complain some ISO 


registered companies believe that if they are ISO registered that only limited DoD 


inspection is required. Should DoD QARs accept ISO registration as certification for the 
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Preaward Quality Assurance Survey and has registration reduced and/or eliminated DoD 


on-site quality inspections or audits during the surveillance phase, should it? 


a. 


Discussion 


This question was posed to address one of the major problems associated between 


the DCMA, industry, and registrars of ISO. The questions goal was to bring into context 


the Third-party registrar and DCMA role in quality assurance audit and surveillance. 


b. 


Responses 


I agree the Preawards could be eliminated, and yes the on site quality 
inspections should be reduced. 


The surveillance audits are quick passovers to check for system 
breakdowns. To fully evaluate a system as critical as ours requires detailed 
surveillance. We have a unique system in that our customer participates on 
our monthly Internal Audit Reviews thereby maintaining oversight 
without performing additional disruptive activity. 


Should DoD QARs accept ISO for the preaward? Since preaward implies 
no final selection has been made, yes. After a preaward is conducted (for 
example, a desk audit) and before signing contracts, it is always good 
business to visit the site and ascertain its true capabilities. That would be 
a pre-contracting audit and should be performed. 


Has ISO certification resulted in a reduction of on-site customer audit 
personnel? In our case, no. It should be, though. As a company matures 
into the ideals surrounding ISO and continuous improvement efforts prove 
to reduce the risk to the customer well below the noise level it would seem 
to be a waste of money to maintain large, on-site oversight personnel. 


The areas (or elements) assessed by the 3 party should not be re-audited 
by DOD. However many DOD evaluations go beyond the ISO 
requirements. ...DOD should evaluate the delta; otherwise an added 
burden has been placed on the contractor. 


I do not think it should be based on ISO registration. We should be able to 
prove to our customer the quality of the product. We should be able to 
conduct an adequate job of monitoring and continually improving the 
measurement of our process, whether or not we use ISO, MIL-Q or any 
other system, that should be their (DCMA) justification for backing off on 
the amount of oversight and surveillance. The system is not as important 
as how we measure up to our customer’s expectations and requirements. 
It is a matter of trust based on data. 


The amount of surveillance or oversight should not be based on any 

redundancies found between registrars and DCMA but driven more by 

how well the process is working and measuring up to the oversight DCMA 
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feels they need to conduct and as to how we are evaluated to contractual 
performance measures. The relationship, if one exists, is if ISO has 
helped us improve and organize our process to a point to where DCMA is 
more confident, not the ISO registration. It is a matter of confidence and 
risk associated with the contract not a correlation of registrars and DCMA. 


I don’t think ISO alone has reduced inspectional and oversight. I believe 
low-budgets, past-performance, fewer resources, ISO, and our good 
relationship with DCMA has reduced surveillance and_ inspection. 
PROCAS and our partnering with DoD have also reduced the amount but 
not ISO alone. 


There are differences among registrars and differences among DCMA 
auditors. Government representatives come from different agencies, from 
various areas and communities, with varying objectives against the same 
contract. DoD agencies waste time with the infighting that occurs 
between the various DoD stakeholders. It constrains DCMA and all the 
other DoD agencies to adequately evaluate an organization because DoD 
cannot reach consensus on what they expect of the contractor performance 
against the contract. The roles of contract administration are not agreed 
upon between DCMA and the buyer. 


The PCO and ACO must clearly designated and define the areas of 
contract administration and procurement functions. The resident DCMA 
representatives should take the lead in deciding the jurisdiction of the PCO 
and the ACO in many of these matters. 


Question 13. (DoD and Contractor Quality Assurance Relationship) 


How should the preaward survey and the follow-on quality surveillance of DoD 


QARs be conducted (consider training, DoD culture changes, and in-plant conduct)? 


a. 


Discussion 


This question was asked as a follow up to the previous question to solicit 


comments on how best, in industries’ view, to address issues raised in Question 11. 


b. 


Responses 


If there is a preaward focus on the performance spec of the product not the 
quality system. QARs should be invited to the management review 
meetings to understand how the quality system is performing and this 
would be an opportunity to provide customer input. 


You establish the expectations before award. Validate a firm 
understanding of how you, as the customer, expect things to be done. 
Validate that those materials being purchased can be assembled or 
manufactured IAW the requirements. Validate the beginning of 
production through thorough audits of the processes. Then establish the 


115 


14. 


plan for follow on visits, while letting the supplier know that any issues 
that arise will bring your group in as a reactionary force to assist with 
issue resolution. While this relationship is being established, you must 
also express that you have a shared interest in the success of the program 
and are willing to do whatever is necessary to insure that success. 


I see no change in the procurement assessment process other than using 
(and not duplicating) the ISO registration as an accredited certificate that 
the Quality Management System is acceptable. This should not affect 
other parameters of pre/post award assessment of suppliers. 


It depends on how DoD chooses to use DCMA. DCMA are not registrars 
and registrars are not DCMA. Registrars are system oriented and conduct 
a paper work audit where DCMA is more hardware or product oriented 
and conduct audits to that degree. Both organizations depend a great 
degree on the skills and knowledge of the person and not so much the 
organizations mandate. It is difficult to find people with both the skills 
and ability to look at both systems approach and the process and product 
approach results. 


The preaward survey is adequate for its intended use. DoD agencies must 
cooperate better to make the system work more effectively. 


Question 14. (DoD and Contractor Quality Assurance Relationship) 


In your view has the DoD culture and approach changed in quality assurance 


audit procedures under accepting ‘most effective quality system’ instead of MIL-Q-9858? 


a. 


Discussion 


The culture of DCMA from an industry viewpoint is addressed in answering this 


question. The question was to gain perspective on outsiders’ view of DCMA culture and 


attitudes effecting MIL-Q to ISO change. 


b. 


Reponses 
The QARs appear to have not changed at all. 


Yes, overall the transition by DOD to the “industry standard” philosophy 
is in place. 


Our relationship with DCMA is better now than it has been in the past. 
The process improvements have been good. But as DCMA reduced their 
personnel by two-thirds over the past decade all you had left was older, 
more senior employees. These employees come from a past culture of 
inspection and monitoring that we (DCMA) cannot trust the contractor. 
You reach a point where it is very difficult for DCMA to change the 
culture. It is a generational issue. They (DCMA) are being taught, trained 
and told to do it differently but can’t or won’t change due to the past 
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cultural influence. The environment has become increasingly more 
technical with change occurring more rapidly than in the past. With the 
high cut-backs in personnel and few new people it is difficult for DCMA 
to attract new people with the qualifications and skills required to 
adequately evaluate an organization from a systems and _ technical 
approach. 


The change is for the better. The culture is focusing more to the processes 
associated with quality than the end result inspections of the past. 


Question 15. (DoD and Contractor Quality Assurance Relationship) 


How effective (strength and weaknesses) is the quality assurance/management 


audit and later surveillance performed by ISO Registrars and DCMA or DCAA Quality 


Assurance Representatives? 


a. 


Discussion 


This question focused on the audit procedures, rather than the auditors, and the 


audits strengths and weaknesses. 


b. 


Responses 


The ISO registrars are reviewing the system every six months; this 
frequency is greater than the QARs review. My opinion is the QARs 
would wait for a nonconformance to occur and then review a process, 
which is to late. The ISO registrars are more proactive. 


Not nearly as effective as our own internal reviews. They lack the 
understanding of the processes and cannot therefore adequately judge 
them. They are not intended to identify all issues. (DCMA) audit can and 
should only be cursory in nature. The supplier must earn trust. (Supplier 
trust)... provides the confidence necessary to the customer. It is the 
supplier’s responsibility to prove that risk is minimal and good value is 
being delivered in each case. The systems, which are implemented, to 
comply with ISO are but one step in creating that assurance. 


Our DCMA auditors perform joint audits with us, as a single team. I 
believe that the DCMA participation in this effort is strong. 


The registrars are head-and-shoulder above DCMA on their ability to 
evaluate compliance to the ISO Standard. They (ISO) are much more 
capable to understand the systems evaluation of an organization and the 
organizations’ written procedures to match the system. Registrars look at 
the entire business system. Registrars are not very good at looking at a 
product and measuring its conformity and compliance to the procedures. 
DCMA is hardware oriented and much more familiar with the hard ware 
requirements. 
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The biggest disconnect is between the DCMA and the system user. More 
needs to be done by DCMA to better understand and determine what the 
user is looking for. This places the contractor as the3d party police officer 
between the DCMA and the user. The DCMA lacks insight from the user 
and the user’s requirements and how to meet the user’s expectations. 


Question 16. (DoD and Contractor Quality Assurance Relationship) 


What changes in training should DoD make to better address ISO 9000:2000 and 


better address the quality assurance needs of industry and DoD customer demands? 


a. 


Discussion 


This question was meant to solicit input from industry on how best to train 


DCMA representatives conducting quality assurance auditing and surveillance. 


b. 


Responses 
Provide training to the QARs for the new standard 


I think the DoD made the correct decision when they decided to get out of 
the standards business and allow industry to run and maintain the system. 


Perhaps sending QARs to the same training used in industry is a good 
suggestion? The American Society for Quality has excellent training 
programs for auditors and technicians that would benefit the QARs. 


I think that DoD should have their auditors trained by RAB certified 
trainers (just like most of the industry does), and not receive their training 
from Government sources. 


It seems to me DCMA is pushing away from the hardware and product 
audit and focusing more on the systems approach. DCMA should stay 
away from this when ISO has registered an organization since ISO looks 
at the quality or management system overall. DCMA should remain 
focused on the hardware and products. DCMA is the representative of the 
military customer and user and should focus on the product. 


DCMA focus should be the procurement and risk management of the 
contract. DCMA is a tool for the Government to use to control the risk 
went the Government buys something. The question is how does the 
Government want to use DCMA. ISO as a tool determines if an 
organization has the capability from a systems point of view. DCMA 
primarily inspects hardware once the order is in and not a very good tool 
to determine the future risk of the supplier. DCMA seems more effective 
after the market research is conducted and a supplier is selected. DCMA 
uses in process and final inspection to compare the product against 
specifications. This type of inspection is better on tank tread than on a 
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missile guidance system. How to use DCMA depends on the product and 
how best to mitigate risk. 


Continue ISO and current industry quality assurance training. I see 
DCMA weakness is their inconsistent approach between the product 
quality and the quality of the entire organizations systems and processes. 
ISO looks at a whole product line. DCMA must take a more process 
oriented view because all our process are finally linked to the product. 
DoD organizations could become Third-party ISO registered 


Question 17. (DoD and Contractor Quality Assurance Relationship) 


How is the cost associated with becoming ISO 9000 certified applied to industry 
cost of doing business (CAS and GAAP)? 


a. 


Discussion 


Current CAS does not allow industry to directly charge ISO QMS application to 


its contracts. The question was asked to garner responses of how does industry apply the 


ISO Registration and QMS to that of doing business. 


b. 


Responses 


From a business perspective one would be hard pressed to associate the 
initial outlays to the benefits over time, although estimates could be put 
together for ROI. Most businesses considering ISO certification are doing 
so because of a need expressed by their customers and ertification is 
pursued to ensure that a competitive edge is maintained, or simply to 
maintain their current business. In many cases a business can find itself 
noncompetitive in international or certain domestic markets if it does not 
have and maintain a certified quality system. 


The cost of registration is an overhead expense and is allocated across the 
entire business. (I am not familiar with CAS/GAAP) 


The fee for registration is a significant amount of money to give to the 
registrar. ISO cost is expensive and that expense is passed down, in one 
form or another, in overhead or what the customer pays. It goes to profit, 
an organization must cover their cost and it will end up somewhere in the 
profit line whether it is covered as overhead or as another cost. 
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C. DCMA INTERVIEW RESPONSES 
1. Question 1 (ISO 9000:2000) 


How is DCMA changing their approach to quality assurance and management 
certification and monitoring due to the changes in ISO 9000 1994 to 2000, as it applies to 


higher-level quality standards and risk management associated in this area? 
a. Discussion 


This question was designed for the respondents to provide insight to the reader on 


the changes DCMA is making in addressing ISO changes. 


b. Responses 


e The DCMA approach has not significantly changed. The 2000 ISO 
versions are seen and addressed as any other higher-level quality 
requirement. DCMA is currently reviewing its practice of issuing 
statements of qualification regarding supplier higher-level quality 
compliance with the intent of deleting the practice, however the 
determination is not based on the ISO change, but rather on the DOD 
practice of not requiring certified systems. 

e DCMA Headquarters view is (ISO 9000) 2000 is not significantly 
different from (ISO 9000) 1994. The civilian sectors see this differently 
and are committing resources to make the change. DCMA should follow 
suite, at least partially, to assure our auditors are able to perform an audit 
at an appropriate level of knowledge and expertise. 


2. Question 2 (ISO 9000:2000) 


Have the problems, shortcomings, and lessons learned with the previous ISO 


9000 standards and registration been addressed under ISO 9000:2000? 
a. Discussion 


This question was used to allow organizations to provide the researcher with 
insight into what DCMA quality assurance practitioners saw as the strengths, 


weaknesses, and opportunities of both ISO 9000:1994 and 2000 provide. 


b. Responses 


e Both respondents believed the new version addresses shortcomings with 
customer satisfaction and continual improvement but introduces new 
problems and concerns due to the cancellation of ISO 9002 and 9003 for 
industry, registers and compliance activities. DCMA plays a unique role 
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when addressing the shortcomings of any QMS. As the frontline worker 
they are enabled to address issues of this type. DCMA, as an in-plant 
representative, is the ombudsman for the Governments quality assurance 
and management program applied to procurement. As the frontline 
ombudsmen the DCMA representative protects Government interests in 
product assurance matching customer requirements with production 
challenges face by industry. 


Question 3 (ISO 9000:2000) 


Will their be any changes to the FAR, DoDD 5000, and to DCMA preaward 


surveys and surveillance? If so, can you provide an overview or any documents showing 


what these changes might be? 


a. 


Discussion 


The question was asked to update any recent changes to DoD guidance not 


covered under Chapter HI. 


b. 


4. 


Responses 


Unaware of any changes proposed solely due to the transition. A FAR 
change was contemplated to introduce ISO 9001-2000, however the 
current version of the FAR does not date versions so the 2000 version is 
already contained. A FAR change has been drafted that will include the 
designation of ANSI/ISO/ASQ Q9001 as a_ higher-level quality 
requirement. This change intended to assure that the accepted American 
version of ISO 9001 is included. 


We are currently working on an ISO 9000-2000 checklist in DCMA. Our 
checklist is nearly identical to most registrars except for about 10% to 
20% of the document. In reviewing many registrars checklist we say that 
the majority of the documents was the same. The similarities are in the 
“shall” and “will” clauses of the registrars’ checklists. The remainder of 
the registrars’ checklist focused on their interpretation of the “may” and 
“should” clauses of ISO. The DCMA ISO 9000-2000 checklist is based 
solely on the “shall” and “will” clauses. DoD cannot contractually hold an 
organization to the “should” clauses unless they are specified outside the 
of the ISO 9000:2000 requirement. 


Question 4 (ISO 9000:2000) 


What planning, guidance, and training is DCMA conducting to address the ISO 
9000:2000 changes? 
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a. Discussion 


This question was used to ascertain the particular local and agency wide training 
on ISO 9000:2000. 


b. Responses 


e The change has been introduced to the workforce and the deltas between 
the 1994 and 2000 versions provided. Under a decentralized training 
environment, introductory training has been provided to a number of 
assigned personnel. Additionally, local organizations are encouraged to 
obtain more intensive training as required. 


e Training is limited. Funding is limited to nonexistent depending on the 
command. CMO’s are told to fund training for ISO from their 
discretionary budget. The focus is on delta training to get the most effect 
for the limited resources for training. The training philosophy across 
DCMA organization to get our workforce knowledgeable on the 2000 
version is “train the trainer.” Many commands are sending one or two 
people to training and in-turn they are responsible for training the people 
in their local organization. There are some weaknesses in this approach 
but given the current resource constraint it is the current course of action. 


e Under the MI-Q 9858 and 45208, each came with a handbook, H50 and 
H51 respectively, on how to properly use the standard. DCMA publishes 
no such handbook for ISO, nor does the IOS. IOS mandates what the 
standard is, not how to implement or evaluate. 


3. Question 5 (ISO 9000:2000) 


From your perspective what changes in ISO 9000 provide the greatest advantage 


and/or disadvantages to industry and the DoD procurement? 
a. Discussion 


This question was asked to establish DCMA respondents opinion on the 


advantages/disadvantages of ISO 9000:2000 
b. Responses 


Overall opinion was the introduction of exclusions to the requirements contained 
in clause 7 provides both the greatest advantages and disadvantages to both industry and 
the Government. The concept allows for the true commercialization of the QMS; 


however industry is responsible for truly understanding their markets and customers and 
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the DoD is now responsible for understanding the exclusions principles and determining 


if exclusions are appropriate, both in the pre and post contract award phases. 
6. Question 6 (ISO 9000:2000) 


How does or what role does customer satisfaction and contractor past 
performance play in DCMA quality assurance and management? What are its problems 


and how could it be made better? 
a. Discussion 


DCMA has the unique role of providing the most interfaces with industry as the 
DoD representative; this question was used to gain DCMAs perspective on that role as it 


pertains to customer satisfaction. 
b. Responses 


Customer satisfaction and supplier past performance serves a great role in DCMA 
QA activities. Risk management activities are extremely reliant on these factors. 
Problems revolve around communications and Government reluctance to export past 


performance information, at; I believe the insistence of industry. 
7. Question 7 (ISO 9000:2000) 


Should the redundancies between ISO 9000 registration and periodic review 
compared to DCMA preaward survey and surveillance be eliminated? Why or why not 


should Government evaluate the redundant areas? 
a. Discussion 


This question was posed to address one of the major problems associated between 
the DCMA, industry, and registrars of ISO. The questions goal was to bring into context 
the Third-party registrar and DCMA role in quality assurance audit and surveillance and 


provide DCMA responses to Industry Question Number 11. 
b. Responses 


Redundancies should be reduced; DCMA policy requires that first, second and/or 
third party audit results be assessed and used prior to performance of redundant activities. 


I do not believe that DCMA activities relative to suppliers QMS are redundant. A case 
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possibly can be made that preaward surveys regarding QA capabilities are not necessary 


for registered suppliers, but that registration should be reviewed in lieu. 
8. Question 8 (ISO 9000:2000) 


Has the SPI and the move to best commercial standards, like ISO, provided the 
DoD a better product and made the acquisition process better for customer, DoD 


manager, and industry? 
a. Discussion 


This question was used to gain the past experience of the DCMA interviewees 


and draw some conclusions about effectiveness of SPI. 


b. Responses 


e The SPI of only one standard to industry is deceiving. MIL-Q and MIL-I 
were not canceled. They are not the preferred method according to DoD 
policy. So when we (DoD) stopped dictating what quality system to use to 
allow the “most effective quality system” we opened the door for industry 
to choose the system. Many organizations decided to remain with the 
MIL-Q and MIL-I standards. DCMA now must not only understand ISO 
9000:2000 but all of the previous Military standards still in use, many 
industry accepted quality standards like QS 9000 the automobile industry 
uses, as well as, ISO 9000:1994. The way the FAR is written we are 
discouraged to dictate to an organization any of these systems as long as 
they meet the “most effective” criteria. The challenge of understanding all 
of these systems, although there are many similarities between them, is 
difficult for DCMA to consistently and appropriately manage across the 
numerous sectors of business DoD is contracted with. 


e Theoretically it makes the acquisition process better. 


e DoD suppliers’ complaint about the cost of having two or more systems 
prior to SPI is only a half-truth. Most organizations only used one system, 
MIL-Q, the cost of quality is something an organization will commit to 
even if it is not contractually outlined. The cost of implementing ISO, if 
they chose to leave MIL-Q, is only a 5% to 10% increase. The similarity 
between ISO 9000-1994 and MIL-Q 9858 were not significant enough, in 
my opinion, to warrant significantly more cost. Suppliers wanted DoD 
inspectors out of their factories and in SPI industry saw a way to 
accomplish the elimination of in-house DCMA representatives. Industry 
and NATOs push b use ISO instead of the Allied Quality Assurance 
Procedures (AQAPs) pushed DoD to make the change. 
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9, Question 9 (Quality System Versus Quality Product) 


One criticism of ISO 9000 registration is it provides no guarantee of product 
quality or conformity, only that a quality process is in place. Do you agree, disagree with 


this statement? 
a. Discussion 


This question is based on the recent incidents of ISO 9000 registered 
organizations producing non-conforming and defective material. Its purpose was to elicit 
comments on some of the problems or misconceptions about ISO registration and what 


measures can be taken to increase quality control and product conformance. 


b. Responses 


e There are good registrars and poor ones. Some will register any 
organization given the right price. Most do not and do conduct a good 
quality system Third Party registration against ISO. What Third Party ISO 
registration does for us (DCMA) is to provide a certain level of confidence 
in the suppliers system we can use to assess the risk of the potential 
contract and make an evaluation on how to precede with the DCMA 
preaward survey. Registrars look at the quality system as a whole DoD 
should not only look at the total system but below that level as well. 
DCMA must inspect or provide insight into the product process quality. 
We track forward and back on product development. 


e There are no absolute guarantees that either will result in a quality 
product. They should assist in identifying any system discrepancies that 
may lead to sub-standard product quality. 


10. Question 10 (Quality System Versus Quality Product) 


What quality assurance/management measures does DCMA take to insure a 
supplier demonstrates product conformity and/or critical characteristics, in terms of 


defect levels, are acceptable; and that improvements are planned and later demonstrated? 
a. Discussion 


The same question type was asked of industry to see what additional measures are 


used to provide product quality assurance. 
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b. Responses 


DCMA employs a risk management philosophy of identifying key processes and 
their potential risks to performance, cost and schedule. Surveillance techniques, 
particularly regarding performance risk, include assessments of QMS, analysis of 


processes, audits of products and ongoing data analysis. 
11. Question 11 (DoD and Contractor Quality Assurance Relationship) 


Has ISO 9000 become the quality management standard for DoD contractors, i.e., 
must a contractor become ISO 9000 certified to conduct business with the DoD 


especially in contracts requiring a higher-level quality standard? 
a. Discussion 


The issue the researcher wanted to raise in this question is, although ISO is not 
mandated by any Federal procurement regulation and guidance, has ISO influence 


created an environment in which industry must be compliant with ISO to conduct 


business. 
b. Responses 
e Under A76 the services have, in-essence, become responsible again for 


contract administration. Under an A-76 program the local procuring office 
administers the contract. Local procuring agencies use local policy. The 
policy is usually based on Army, Navy, or Air Force Regulation and 
Guidance, not ISO standards. An organization like DynaCorps Inc. who is 
ISO Third Party Registered and accepts an A76 still has to be judged 
against the local policy for award and later for surveillance. Some local 
policy has adapted ISO. Service members and local contract officers have 
trained to conduct ISO Third-Party Registration. One example is DSC- 
Columbus. A-76 is another product of inconsistency of quality assurance 
and management across DoD faced by industry. 


12; Question 12 (DoD and Contractor Quality Assurance Relationship) 


Should DoD impose ISO 9000 standards and registration on contractors and sub- 


contractors as an entrance to conducting business with the DoD? 
a. Discussion 


This question was meant to elicit general comments on the benefits and problems 
of a DoD mandate of ISO 9000. 
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b. Responses 


It was the opinion of the DCMA respondents that ISO should not be imposed. 
DCMA as well as industry believed that it will limit competition and may force out 


smaller entities. Further analysis is providing under Industry Question Number 10. 
13. Question 13 (DoD and Contractor Quality Assurance Relationship) 


Some DoD Quality Assurance Representatives (QARs) complain some ISO 
registered companies believe that if they are ISO registered that only limited DoD 
inspection is required. Should ISO registration be accepted by DoD QARs as certification 
for the Preaward Quality Assurance Survey and has registration reduced and/or 
eliminated DoD or-site quality inspections or audits during the surveillance phase, should 
it? 

a. Discussion 


This question was posed to address one of the major problems associated between 
the DCMA, industry, and registrars of ISO. The questions goal was to bring into context 


the Third-party registrar and DCMA role in quality assurance audit and surveillance. 


b. Responses 


e A case may be made to limit preaward surveys of companies that have 
certification; I believe more study is needed. I don’t believe that ISO 
registration has been responsible for reduced QA activities. It’s my 
opinion that QA activities on-site have been reduced due to personnel and 
money constraints forcing “smarter” decisions at both the pre and post 
award phases to conc entrate activities based on risk. 


e Earlier each service had their respective folks doing contract 
administration, i.e, AF PROS, and NAV PROS. We consolidated 
resources and became DCMC under DLA. We recently became our own 
agency and no longer fall under DLA. Our early auditors used the 8200.1 
“Black Book.” The “Black Book” spelled out how to conduct quality 
audits. The book used defined and express verbiage for many contracting 
scenarios and issues. We use product evaluations (PEs) and Product 
Verification Inspections (PVIs) in the conduct of contract administration. 
The 1990’s saw the end of this type of inspection process and ushered in 
insight instead of inspection. The “One Book” replaced the “Black Book” 
and with it went specifics to generalists. DCMA went from a stovepipe 
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14. 


organization of the Quality Directorate, the Engineering Directorate, and 
the Contracting Directorate to a teaming approach. The approach does not 
work well and comes with its own set of problems. Under the stovepipe 
approach DCMA was able to mandate and control policy under each of the 
directorates. The team leader now controls the agenda of the team. The 
team leaders are not usually trained in all the functions so what directorate 
the team leader came from is the main agenda of the team with the other 
areas usually suffering due to the teaming approach. Under the new 
teaming system a great deal of latitude is given to each DCMA command 
and even office to define quality management and assurance according to 
current contract preaward surveys. The approach causes _ great 
inconsistency between how DCMA conducts audits and inspection, not 
only regionally but also with-in the same office. 


Question 14 (DoD and Contractor Quality Assurance Relationship) 


In your view has the DoD culture and approach changed in quality assurance 


audit procedures under accepting “most effective quality system’ instead of MIL-Q-STD 


9858? 


a. 


Discussion 


The culture of DCMA from an employee’s viewpoint is addressed in answering 


this question. The question was to gain perspective on an insiders’ view of DCMA 


culture and attitudes effecting MIL-Q to ISO change. 


b. 


Responses 
No, not particularly, due to the “most effective quality system” concept. 


In 1988 DoD wanted nothing to do with ISO. The culture, while slowly 
changing, still has yet to wholly embrace the SPI ideals around ISO. 
Culture is difficult to change. The aging of our workforce, the decline in 
our quality assurance workforce numbers from 24,000 in 1995 to 12,000 
currently (numbers are approximate) and our inability to attract some of 
the best talent in the field of quality assurance and management are factors 
affecting the DCMA culture. 
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